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The  goal  of  this  paper  is  to  demonstrate  that  a  method  for  inductive 
program  synthesis  (as  described  in  Schmid  &  Wysotzki  1998)  can  be 
applied  to  the  problem  of  learning  cyclic  (iterative/recursive) 
macro-operations  from  planning.  Input  in  the  program  synthesis  system 
is  a  so-called  initial  program  which  represents  an  ordered  set  of 
straight-forward  transformations  from  input  states  to  the  desired  output.  In 
the  context  of  planning,  the  input  states  correspond  to  initial  states,  the 
output  state  to  the  planning  goal,  and  transformations  are  shortest 
operation  sequences.  Ordering  of  transformations  can  be  achieved  by 
calculating  a  minimal  spanning  tree  for  the  problem  graph  with  the 
state(s)  fulfilling  the  goal  as  root.  We  have  implemented  a  non-linear 
backward  planner  which  generates  such  a  complete  partial  order  as  a 
by-product  of  planning.  Output  of  the  program  synthesis  system  is  a 
recursive  program  scheme  representing  the  generalization  of  a  program 
limited  to  solving  a  finite  problem  of  given  size  to  a  general  solution 
strategy.  Our  synthesis  method  is  embedded  in  the  theory  of  the 
semantic  of  functional  programs  and  in  the  theory  of  inductive  inference 
(see  Muehipfordt  &  Schmid  1998)  and  thereby  provides  a  sound  formal 
basis  for  macro-construction.  The  current  implementation  can  generalize 
tail,  linear  and  tree  recursive  structures  and  combinations  of  such 
structures  with  multiple  (and  possibly  interdependent)  recursive 
parameters.  89  pages 


Abstract 


The  goal  of  this  paper  is  to  demonstrate  that  a  method  for  inductive  program 
synthesis  (as  described  in  [SW98])  can  be  applied  to  the  problem  of  learning 
cyclic  (iterative/recursive)  macro-operations  from  planning.  Input  in  the  pro¬ 
gram  synthesis  system  is  a  so-called  initial  program  which  represents  an  ordered 
set  of  straight-forward  transformations  from  input  states  to  the  desired  output. 
In  the  context  of  planning,  the  input  states  correspond  to  initial  states,  the 
output  state  to  the  planning  goal,  and  transformations  are  shortest  operation 
sequences.  Ordering  of  transformations  can  be  achieved  by  calculating  a  mini¬ 
mal  spanning  tree  for  the  problem  graph  with  the  state(s)  fulfilling  the  goal  as 
root.  We  have  implemented  a  non-linear  backward  planner  which  generates  such 
a  complete  partial  order  as  a  by-product  of  planning.  Output  of  the  program 
synthesis  system  is  a  recursive  program  scheme  representing  the  generalization 
of  a  program  limited  to  solving  a  finite  problem  of  given  size  to  a  general  solu¬ 
tion  strategy.  Our  synthesis  method  is  embedded  in  the  theory  of  the  semantic 
of  functional  programs  and  in  the  theory  of  inductive  inference  (see  [MS98]) 
and  thereby  provides  a  sound  formal  basis  for  macro-construction.  The  cur¬ 
rent  implementation  can  generalize  tail,  linear  and  tree  recursive  structures  and 
combinations  of  such  structures  with  multiple  (and  possibly  interdependent) 
recursive  parameters. 
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Chapter  1 

Introduction 


Learning  macro-operators  is  a  topic  of  interest  nearly  since  the  beginning  of 
planning  research  [FNTla,  Min85,  Kor85].  While  macros  do  not  lead  in  gen¬ 
eral  to  greater  efficiency  in  planning  [Min855  DC96],  they  are  nevertheless  a 
useful  and  interesting  contribution  to  planning  research.  Firstly,  if  construc¬ 
tion  of  macros  is  not  performed  blindly  but  augmented  by  information  about 
occurrence  frequencies  of  operator  sequences  and  if  retrieval  of  macros  is  orga¬ 
nized  efficiently,  planning  can  be  speed-up  considerably  when  introducing  linear 
macros  [Min85].  A  linear  macro-operator  generally  represents  an  aggregation 
of  the  pre-  and  post-conditions  over  a  sequence  of  primitive  operators.  Using 
a  macro-operator  instead  of  a  set  of  primitive  operators  reduces  the  number  of 
match-select-apply  cycles  executed  during  planning  and  thereby  also  the  number 
of  planning  decisions  which  might  result  in  backtracking.  For  iterative  macros 
efficiency  gains  should  be  even  higher,  because  they  not  only  provide  aggregated 
operators  but  additionally  incorporate  part  of  the  control  strategy  for  planning 
[SC89,  BV96]. 

Secondly,  learning  in  planning  is  per  se  a  topic  of  interest  for  cognitive  sci¬ 
ence  and  artificial  intelligence.  Work  on  linear  macro-operators  in  planning  has 
its  parallel  in  work  on  learning  by  chunking  [RN86]  and  knowledge  compilation 
[And86]  in  cognitive  science.  While  this  work  addresses  the  fact  that  human 
problem  solvers  improve  their  skills  by  experience  (i.e.  faster  generation  of  solu¬ 
tions  which  less  errors  in  accordance  with  the  power  law  of  practice  [NR81]),  it 
neglects  a  second  important  aspect  of  human  learning:  When  solving  a  problem 
of  given  complexity  (as  the  Tower  of  Hanoi  with  three  discs)  humans  can  extrap¬ 
olate  a  general  strategy  for  solving  problems  of  the  same  type  with  arbitrary 
complexity  (as  Tower  of  Hanoi  problems  with  an  arbitrary  number  of  discs; 
[Kla78]).  In  production  system  architectures  with  fixed  interpreter  strategy  this 
kind  of  learning  cannot  be  modelled  [SC89,  SWar].  Therefore,  coming  up  with 
a  technique  for  learning  cyclic  macro-operators  from  experience  is  a  challenging 
problem  for  planning  as  well  as  cognitive  science  research.^ 


^Now  macro  learning  becomes  also  of  interest  in  reinforcement  learning.  See  NIPS’98 
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CHAPTER  1.  INTRODUCTION 


The  importance  of  learning  iterative  macro-operators  was  first  addressed  by 
Carbonell,  Cheng  and  Shell  [CC86,  SC89j  DC96]  and  is  also  acknowledged  by 
Schmid  in  the  context  of  machine  learning  [SW98]  and  cognitive  science  [SWar]. 
While  the  work  of  Carbonell  and  colleagues  addresses  efficiency  gains  in  planning 
and  gives  ideas  about  a  technique  for  constructing  iterative  macros  in  context  of 
the  expert  system  FERMI,  Schmid  proposes  to  combine  planning  and  program 
synthesis  in  the  following  way:  Planning  provides  straight-forward  “programs” 
for  transforming  input  into  desired  output  states.  These  straight-forward  pro¬ 
grams  can  be  generalized  by  a  technique  for  inductive  program  synthesis.  The 
first  step  corresponds  to  exploring  a  problem  or  -  in  the  context  of  program 
construction  —  to  hand-simulating  I/O  transformations.  The  second  step  cor¬ 
responds  to  (unsupervised)  learning  from  experience  and  -  in  the  context  of 
automatic  programming  -  to  programming  by  demonstration  [Coh98,  Wel99a]. 
While  the  first  step  is  dependent  on  background-knowledge  (especially  on  the 
semantics  of  predefined  operations  and  on  knowledge  about  data  structures),  the 
second  step  can  be  performed  (with  some  limitations,  see  [MS98])  purely  syntac¬ 
tically.  That  is,  generation  of  initial  programs  is  dependent  on  heuristic  infor¬ 
mation  -  but  the  synthesis  method  corresponds  to  a  generic  pattern-matching 
algorithm  which  can  be  completely  formalized  and  its  soundness,  efficiency  and 
scope  can  be  determined  in  a  general  way. 

In  this  paper  we  will  focus  on  generating  initial  programs  from  planning. 
The  synthesis  technique  is  described  in  detail  elsewhere  [SW98,  MS98].  Be¬ 
cause  our  approach  is  somewhat  out  of  the  context  of  current  research  in  plan¬ 
ning,  we  start  with  an  informal  example  to  motivate  our  ideas.  More  concrete, 
we  will  show  that  the  function  clearblock  can  be  easily  inferred  from  planning 
experience.  The  clearhlock-innction  is  an  example  of  a  simple  iterative  macro 
[MW87],  corresponding  to  a  linear  recursion.  Afterwards,  we  will  introduce 
our  planning  system  and  discuss  it  in  relation  to  other  planning  systems  as 
PRODIGY  [VCP+95],  UCPOP  [PW92],  graphplan-approaches  [BF97,  KNH97] 
and  universal  planning  [Sch87,  Sch95,  CRT98].  Our  system  DPlan  is  a  sound 
and  complete  non-linear  backward  planner  implemented  in  LISP.  In  the  fourth 
part,  we  will  give  further  examples  of  learning  linear-recursive  macros,  as  for 
example  a  general  solution  strategy  for  the  rocket-problem  with  an  arbitrary 
number  of  objects  [VC93b]  and  afterwards  we  will  discuss  more  complex  exam¬ 
ples  as  constructing  a  tower  of  ordered  blocks  and  Tower  of  Hanoi;  furthermore 
we  will  address  the  application  of  planning  to  list  and  number  problems.  We 
will  conclude  with  an  evaluation  of  our  approach  and  further  work  to  be  done. 


workshop  “Abstraction  and  Hierarchy  in  Reinforcement  Learning”:  Amy  McGovern,  Univer¬ 
sity  of  Massachusetts,  Amherst,  acQuire- macros:  An  Algorithm  for  Automatically  Learning 
Macro- Actions,  and  David  Andre,  University  of  California,  Berkeley,  Learning  Hierarchical 
Behaviors;  http:/ /www-anw.cs.umass,edu/~  dprecup/call _for-participation.html. 


Chapter  2 


How  to  clear  a  block  — 
inductive  generalization  of 
plans 

2.1  A  recursive  function  for  clearing  a  block 

A  human  problem  solver  in  many  cases  is  able  to  generalize  a  strategy  for  solving 
a  complete  class  of  problems  from  his/her  experience  of  solving  some  example 
problems.  For  example,  if  a  person  is  able  to  clear  the  bottom  block  of  a  three 
block  tower,  we  assume  that  he/she  also  is  able  to  clear  the  bottom  block  of  a 
tower  consisting  of  an  arbitrary  number  of  blocks,  even  if  the  person  has  never 
performed  this  action  sequence  before.  The  need  of  automated  mechanisms 
for  this  kind  of  learning  by  experience  was  for  example  addressed  more  than 
a  decade  ago  by  Manna  and  Waldinger  [MW87].  Our  proposal  differs  in  two 
aspects  from  this  seminal  work:  firstly,  while  Manna  and  Waldinger  discuss  the 
synthesis  of  imperative  programs,  we  will  discuss  the  synthesis  of  functional 
programs;  secondly  -  and  more  crucially  -  Manna  and  Waldinger  proposed  a 
deductive  technique  for  synthesizing  conditional  and  recursive  plans  while  we 
are  proposing  an  inductive  technique,  based  on  inductive  program  synthesis. 

The  plan  Manna  and  Waldinger  can  derive  automatically  from  a  goal  spec¬ 
ification  by  their  deductive  tableau  method  is 

{if  clear{a) 

then  A 

else  mak  eel  ear  (hat  [a)); 
put  {hat  {a)  y  table). 

To  clear  a  block  a,  first  determine  whether  it  is  already  clear.  If  not,  clear 
the  block  that  is  on  top  of  block  a  and  then  put  that  block  on  the  table.  If  a 
block  a  is  clear,  the  empty  sequence  of  operations  A  is  returned. 
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CHAPTER  2.  INDUCTIVE  GENERALIZATION  OF  PLANS 


Our  functional  variant  of  this  program  is 

clearblock{x,  s)  =  g(cl€artop{x),  s,puttable{topof{x),  cl€arblock{topof{x),s)). 

We  represent  conditionals  by  the  McCarthy-conditional  g{x,y,z)  =  if  x  then  y 
else  z.  To  clear  a  block  x  in  a  situation  s,  check  whether  x  is  already  clear.  If 
yes,  do  nothing  and  return  the  current  situation;  otherwise,  put  the  block  lying 
on  top  of  block  x  on  the  table  in  a  situation  where  this  block  is  already  cleared. 
The  situation  variable  s  holds  a  conjunction  of  propositions,  as  {on(A  B),  on(B 
C),  clearblock(A)}  where  A,  B  and  C  are  constants.  The  predicate  cleartop(x) 
is  true,  if  x  is  currently  instantiated  with  a  constant  K  and  clearblock(K)  is  an 
element  of  the  current  situation  s.  The  function  topof(y)  is  defined  as  on(x  y)  = 
topof(y)  x).  Application  of  the  operator  puttable(y,s)  has  the  effect  that  on(y 
z)  is  deleted  and  clearhlock(z)  is  added  from/to  the  propositions  representing  sit¬ 
uation  5.  The  function  clearblock(x,s)  is  linear  recursive,  that  is,  it  corresponds 
to  a  loop  or  -  in  other  words  -  to  an  iterative  macro- operation.  In  contrast  to 
the  iterative  macros  proposed  by  Shell  and  Carbonell  [SC89],  in  our  case,  if  the 
precondition  is  fulfilled  the  macro  terminates,  otherwise,  the  operation  puttable 
is  (repeatedly)  applied. 

Excursion:  Inductive  synthesis  of  recursive  program  schemes 

We  will  present  the  general  idea  of  our  synthesis  method  in  a  nutshell. 

The  method  was  originally  proposed  by  Wysotzki  [Wys83]  as  an  extension 
of  Summers’  method  [Sum77]  and  was  extended  further,  implemented 
and  formalized  in  the  framework  of  grammar  inference  by  Schmid  fSWQS, 

MS98]. 

The  synthesis  technique  starts  with  an  initial  program  as  input  and 
tries  to  fold  it  by  extrapolating  a  recursive  program  scheme.  An  ini¬ 
tial  program  is  a  nested  conditional  expression  representing  an  ordering 
of  straight-forward  transformations.  We  can  synthesize  the  clearblock- 
function  from  the  following  initial  program: 

G  =  g{cl€artop(x)ySjputtable{topof{x), 

g{deartop{topof{x))yS,puttable{topof{topof(x))^ 

my 

This  initial  program  represents  the  experience  of  clearing  a  block  in  a 
world  consisting  of  maximally  three  blocks.  If  there  are  more  blocks  (i.e. 
topof{topof{x))  is  not  clear),  the  actions  are  undefined  (Q).  Note,  that 
the  initial  program  corresponds  to  the  second  unfolding  of  the  clearblock- 
fimction  given  above.  That  is,  for  program  synthesis  we  reverse  the  idea 
of  determining  the  semantic  of  a  recursive  fimction  as  its  smallest  fixpoint 
[FH88]:  from  a  given  sequence  of  unfoldings  we  want  to  extrapolate  the 
minimal  recursive  program  scheme  which  can  generate  these  unfoldings. 

An  initial  program  G  can  be  folded  into  a  recursive  program  iff  it 
can  be  decomposed  into  a  sequence  =  Q,  /m)  with 

/  =  1 . . .  =  G  of  partial  transformations  which  successively  cover  a 

larger  amount  of  inputs.  That  is,  we  have  to  find  a  term  tr  in  G  which  has 
the  characteristic  that  successively  more  complex  subterms  of  G  can  be 
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expressed  by  inserting  the  previous  subterm  at  position  m  where  variables 
V  are  substituted  by  terms  t.  For  our  example  we  have: 

=  n 

g^^^  =  g(cleartop{x),  s,puttable{topof{x),Q)) 

g^^^  =  g(cleartop(x)j  s,puttable(topof(x), 

g(cleartop(topof(x))j  s,puttable(topof(t;opof(x)),Q)))) 

=  G 

with  tr  =  g(cleartop(x),  s,  puttable(topof(x),  m))  and  the  substitutions 
[topof{x)  lx]. 

Because  g^^^  =  Im)  holds  for  all  ^’s, 

=  n 

=  g[cleartop{x),s,puttaUe{topof{x),g^l^^^^^y^.^)) 

g^^'>  =  g{cleartop{x),s,puttableitopof{x),gl2pof{x)/a:])) 

we  can  fold  G  into  g  =  tr(g[t/v]/m).  For  our  example  that  is 
g  =  g{cleartop{x),s,puttable{topof(x),g[topof{x)/3:])) 

which  corresponds  to  the  clearblock-prograxa  given  above. 

Our  method  is  defined  independently  of  a  given  programming  lan¬ 
guage.  Instead,  we  consider  programs  as  terms  which  are  elements  of  some 
arbitrary  term  algebra  and  we  infer  recursive  program  schemes  [CN78] 
which  can  be  interpreted  with  respect  to  a  programming  language.  With 
oim  method  we  can  infer  tail  recursive  structures  (i.e.  for-loops),  hnear 
recursive  structures  (i.e.  while-loops),  tree-reciusive  structures  and  com¬ 
binations  thereof.  We  can  deal  with  programs  starting  with  a  constant 
part  cind  -  what  is  beyond  the  scope  of  ILP-approaches  [MDR94,  FYar]  - 
we  can  deal  with  multiple  recursive  parameters  which  can  be  interdepen¬ 
dent.  What  is  currently  out  of  the  scope  of  our  method  are  substitutions 
which  are  themself  performed  recursively  (as  in  the  ac/cer man-function). 

For  details  about  the  formal  background,  the  synthesis  algorithm,  its  scope 
and  complexity,  see  [SW98,  MS98]. 

Our  method  of  program  synthesis  from  initial  programs  can  be  seen  as  an 
approach  to  learning  by  generalizing  over  experience  or  -  in  the  context  of 
programming  -  as  programming  by  demonstration.  From  the  perspective  of 
planning  research,  we  argue  that  planning  provides  initial  experience  with  a 
problem  (i.e.  a  kind  of  initial  program)  which  we  can  generalize  by  means  of 
program  synthesis.  That  is,  we  propose  a  method  which  makes  it  possible  to 
infer  a  large  class  of  cyclic  macros  in  a  formally  sound  way.  From  the  perspective 
of  program  synthesis  research,  we  argue  that  the  method  to  construct  initial 
programs  by  rewriting  input/output  examples  (see  below)  can  be  replaced  by 
more  powerful  (and  more  natural)  planning  techniques.  To  sum  up,  we  propose 
the  following  steps  for  learning  cyclic  macro-operators: 
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CHAPTER  2,  INDUCTIVE  GENERALIZATION  OF  PLANS 


1.  Constructing  initial  programs  from  input/output  examples: 

(a)  Construct  an  ordered  set  of  straight-forward  transformations  from 
input  states  to  output  (i.e.  goal)  states  by  planning 

(b)  Transform  the  plan  into  an  initial  program 

2.  Generalizing  initial  programs  into  cyclic  macro-operations  by  program 
synthesis. 

Above,  we  showed,  how  the  cyclic  macro  for  the  clearblock-piohlem  can  be 
inferred  from  an  initial  program.  In  the  rest  of  this  section,  we  will  describe 
the  first  step  -  constructing  an  initial  program  by  planning  -  for  the  clearhlock- 
problem. 


2.2  A  plan  for  cleciring  a  block  in  a  finite  prob¬ 
lem  space 

In  the  following  we  will  show  how  the  initial  program  for  the  clearhlock-pTohleia 
can  be  automatically  constructed  by  planning.  For  the  simple  clearblock- 
problem  we  need  only  one  operator  -putiable  -  which  we  define  in  a  STRIPS-like 
syntax: 

(setq  rules  '(  (rule  puttable 

(if  (  ct  (>  x)  )  (  on  (<  x)  (>  y)  )  ) 

(then  (puttable  (<  x))) 

(conq  (  add  ((ct  (<  y))) 


The  presented  notation  is  the  one  used  for  our  planner  DPlan:  The  precon¬ 
ditions  are  given  as  a  list  starting  with  the  keyword  if  (with  ct  as  shorthand 
notation  for  cleartop).  The  preconditions  are  interpreted  as  conjunction  of  pos¬ 
itive  literals  with  existential  quantified  variables.  Variables  are  represented  by 
(>  a?)  signaling  that  the  binding  for  x  is  to  be  obtained  from  the  current  state 
or  (<  a?)  signaling  that  x  has  to  be  instantiated  in  accordance  with  the  current 
bindings.  The  operator  is  given  after  the  keyword  then.  The  operator  effect  is 
given  as  usual  by  add-  and  del-lists  (which  represent  conjunctions  of  positive 
literals  with  existential  quantified  variables).  Our  planner  can  also  deal  with 
conditional  effects  which  we  will  show  in  the  next  section. 

For  our  example  we  assume  a  blocksworld  consisting  of  three  blocks  A,  B 
and  C.  Our  planning  goal  is 

(setq  goal  ' (  (ct  C)  )  ) . 

DPlan  can  deal  with  goals  which  are  composed  of  a  conjunction  of  (possibly 
interdependent)  subgoals  which  will  also  be  discussed  in  the  next  section. 
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To  be  able  to  generalize,  we  need  to  make  the  following  experience:  clearing 
C  when  it  is  the  bottom  of  a  three-block  tower,  clearing  C  when  it  is  the  bottom 
of  a  two-block  tower  and  clearing  C  when  it  is  already  clear.  That  is,  we  want 
to  regard  for  example  the  following  initial  states: 

(setq  states  ' ( 

((on  A  B)  (on  B  C)  (ct  A)) 

((on  B  C)  (ct  A)  (ct  B)) 

((ct  A)  (ct  B)  (ct  C)) 

)). 


Typically,  a  planner  would  produce  a  separate  plan  for  each  of  the  initial  states. 
In  contrast,  we  want  to  deal  with  ail  three  states  in  a  single  planning  process 
and  thereby  obtain  a  plan  covering  all  given  initial  states^.  The  idea,  to  use 
planning  over  sets  of  states  in  program  synthesis  was  originally  proposed  by 
Wysotzki  [Wys87]. 

Informally,  our  planner  proceeds  in  the  following  way: 

1.  Search  for  all  states  in  states  which  are  subsumed  by  goal  (i.e.  where  the  set 
of  goal  literals  is  a  subset  of  a  state). 

2.  If  there  are  no  such  states:  stop  with  error  (the  goal  cannot  be  fulfilled) 

else  delete  these  states  from  states  and  use  them  as  root  for  the  plan  (If  there 
is  only  one  state  fulfiUing  the  goal,  use  this  state  as  root,  otherwise  introduce  a 
top-node  with  all  states  fulfilling  the  goal  as  children.). 

3.  For  each  node  in  the  plan  do  recursively  until  states  is  empty: 

(a)  Calculate  all  immediate  predecessors  (by  backward-application  of  operator 
effects).  Insert  the  operations  as  arc-labels  and  the  predecessors  as  new 
nodes  in  the  plan^ . 

(b)  Delete  all  states  corresponding  to  the  predecessors  from  states. 

Deleting  all  states  corresponding  to  calculated  predecessors  from  the  set  of  initial 
states  guarantees  that  only  the  shortest  possible  (optimal)  operation  sequences 
to  transform  a  state  into  the  goal  are  calculated  and  that  there  occur  no  cy¬ 
cles.  Furthermore,  the  planner  can  work  completely  without  backtracking.  The 
resulting  plan  is  a  complete  partial  order  over  the  initial  states,  i.e.  the  mini¬ 
mal  spanning  tree  [GH85]  of  the  problem  space  with  the  goal  state(s)  as  root. 
Each  planning  step  is  saved  as  a  structure  of  instantiated  operation,  predeces¬ 
sor  node,  constructed  new  state  (child  node),  and  the  instantiated  preconditions 
and  effects. 

To  generate  (shortest)  operation  sequences  for  transforming  a  set  of  n  initial 
states  into  the  desired  goal,  in  classical  approaches  planning  has  to  be  performed 
n  times.  Each  planning  episode  could  involve  backtracking.  We  are  arguing, 
that  regarding  a  set  of  states  “simultaneously”  in  one  backtracking-free  planning 
process  is  more  efficient  than  planning  n  times  with  backtracking.  Furthermore, 


^  We  will  discuss  the  relation  of  DPlan  to  universal  planning  in  the  next  chapter. 
^This  step  will  be  discussed  in  detail  in  the  next  chapter. 
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Figure  2.1;  Output  of  DPlan  for  clearblock 


by  providing  the  planner  with  a  set  of  states,  there  is  no  need  of  providing  axioms 
for  checking  the  admissibility  of  predecessor  states.  Instead,  we  can  determine 
if  a  new  state  is  legal  by  a  look-up  in  the  set  of  states.  Finally,  if  generating 
predecessor  states  involves  instantiation  of  free  variables  (as  y  in  puttable)  the 
set  of  legal  instantiations  can  also  be  calculated  by  look-ups  in  the  set  of  states. 
In  section  3  we  will  present  these  aspects  of  DPlan  in  detail. 

The  output  DPlan  generates  for  the  clearblock-pTohlenn  is  given  in  figure 
2.1^.  Only  one  state  fulfilled  the  goal  (ct  C).  This  state  is  the  root  of  the 
plan.  Only  one  operation  —  (puttable  A)  —  is  applicable.  Its  backward  appli¬ 
cation  results  in  a  single  legal  predecessor  state.  Again,  only  one  operation  - 
(puttable  B)  —  is  applicable.  The  next  predecessor  cannot  be  expanded  fur¬ 
ther.  All  states  from  states  have  been  used  in  planning.  The  plan  tree  (only 
a  list  for  this  simple  example)  represents  the  shortest  operator  sequences  for 
transforming  each  initial  state  into  the  goal:  ((CT  A)  (CT  B)  (CT  C))  fulfills 
the  goal  already,  the  other  states  can  be  transformed  by;  puttable(A,  ((on  B  C) 
(CT  B)  (CT  A)))  and  puttable(B,  puttable(A,  ((on  A  B)  (on  B  C)  (CT  A)))). 

2.3  Replacing  constants  by  constructive  expres¬ 
sions 

In  the  rest  of  this  section  we  will  describe  how  the  plan  produced  by  DPlan  is 
transformed  into  an  initial  program.  The  crucial  aspect  of  this  transformation 
is  to  replace  constants  by  constructive  expressions.  To  motivate  this  step,  we 

All  graphics  are  the  original  outputs  produced  by  DPlan.  Operations  are  regarded  as  arc- 
labels  but  are  represented  as  intermediate  nodes  between  two  states  in  the  graphical  output 
of  DPlan. 
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make  an  excursion  to  program  synthesis  again. 


An  excursion  to  Summers’  method 

In  Summers’  [Sum77]  classical  approach  to  inductive  program  syn¬ 
thesis,  the  first  step  is  to  rewrite  the  given  input/output  examples  con¬ 
structively,  To  use  Summers’  own  example  (and  notation),  if  we  have  the 
I  /  O-pairs 


()  ^  () 

(^)  ^  ((^)) 

(AB)  ((yl)(S)) 
(ABC)  ^  ((A)(S)(C)) 


the  righthand-sides  (outputs)  are  rewritten  into  functional  expressions 
for  transforming  the  input  into  the  output 


{/iM 

—  nil; 

f2[x] 

=  cons[x;nil]; 

/sM 

=  cons[cons[car[x];nil] 

;  cons[cdr[x];nil]]; 

=  cons  [con  s  [ca  r  [a;] ;  n  il] 

;  cons[cons[ca(ir[a:];  nil] 

cons[cddr[x]\ ni/]]]}. 


Rewriting  is  performed  by  applying  the  predefined  functions  car,  cdr 
their  legal  compositions  and  cons  to  the  input-part  of  the  example  in 
such  a  way  that  they  transform  the  input  into  the  desired  output^.  The 
I/O-examples  can  now  be  written  as 


{() 

(A) 

(AB)- 

(ABC) 


nil; 

cons\x;  m7]; 

cons[cons[car[x\;nU'\cons{cdr[x];niV^; 
con3[cons[car[x\;ni{\cons\cons[cadr[x\.fnil\ 
cons[cc?dr[a:];  nz7]]]}. 


This  corresponds  to  the  minimal  spanning  tree  generated  by  DPlan;  for 
each  state  (input  example)  we  are  provided  with  the  shortest  operation 
sequence  to  transform  the  input  into  the  goal  (desired  output)  and  these 
sequences  are  ordered  by  their  complexity.  We  wiU  show  later,  that  plan¬ 
ning  provides  a  far  more  powerful  method  for  rewriting  I/O-examples  than 
the  technique  proposed  by  Summers. 

After  rewriting  the  output-parts  of  the  examples,  we  need  a  more 
abstract  way  to  discriminate  between  the  different  inputs.  The  equations 
above  express  statements  as  “if  parameter  x  of  function  f{x)  has  value 
‘(A)’  then  return  {cons  x  nz7)”.  Now  we  want  to  replace  the  input  values 
by  boolean  expressions  as  “if  x  is  a  one-element  list”.  To  obtain  this, 

^Summers’  method  was  restricted  to  functions  with  a  single  list  as  argument  which  return 
a  list  by  using  only  the  primitive  operations  car,  cdr  and  cons.  Rewriting  was  restricted  much 
more  than  it  is  the  case  in  genetic  programming  (see  for  example  [Koz92]  for  the  synthesis  of 
the  “tower”  program). 
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Summers  relies  on  a  predefined  complete  partial  order  (cpo)  over  lists 
(with  the  empty  list  as  bottom-element).  He  abstracts  from  the  concrete 
inputs  and  classifies  their  abstract  forms  with  respect  to  the  cpo.  For 
example,  the  form  of  {A  B)  is  {u  lo)  with  (a;)  <  (a;  w)  <  (a;  u;  a;)  can  be 
characterized  by  atom[cddr[x^.  We  omit  the  details  given  in  [Sum77];  for 
our  example  we  obtain: 

pi[x]  =  atom[x\ 

P2{x]  =  atom\cdr[x\[ 

P2>[x]  —  atom[cddr[x]\ 

Pa[x]  =  T. 

The  first  step  of  Summers’  program  synthesis  technique  is  finished 
when  the  p*  -f  fi  expressions  are  combined  into  a  single  function 

^  [Qtom[x]  “)■  nil 

atom[cdr[x^  — >  cons\x\ni!\ 

atom[cddr[x]\  cons{cons[car[x]\nU]\cons[cdr[x]\nU]\ 

T  cons[cons[car[x];nil]\cons[cons[cadr[x'\\nU]\ 
cons[cddr[x]\nil'\\^. 

This  is  the  kind  of  initial  program,  Summers  constructs  from  I/O- 
examples.  In  the  next  step,  the  initial  program  is  folded  into  a  recursive 
program  in  a  similar  but  more  restricted  way  than  our  method  (see  section 
2.1).  By  detecting  regularities  between  succeeding  subexpressions,  the 
following  program  for  “unpacking”  lists  is  synthesized: 

unpack[x]  [atom[x]  — nil] 

T  u[a:]] 

u[ir]  [atom[cdr[x][  cons[x]  nz7]; 

T  -4  cons[cons[car[x]]nil]]u[cdr[x]^. 

To  come  back  to  the  c/earWocAr-problem:  Planning  provided  us  with  (nearly) 
the  righthand-side  of  such  straight-forward  programs.  For  our  clearhlock  exam¬ 
ple  (cf.  figure  2.1)  we  have: 

fi[x,s]  ->  s 

/2[a:,s]  "4  puttable{B,s) 

fsix^s]  putiable{A,puttable{B^  $)). 

In  contrast  to  Summers,  we  still  have  the  constants  given  in  the  states  instead 
of  more  abstract  expressions.  Summers  used  knowledge  about  the  structure  of 
lists  and  a  predefined  order  over  their  complexity  to  rewrite  inputs  as  (A  B) 
into  atomfcddrfx]].  We  will  do  something  similar  to  this  idea:  providing  our 
system  with  background  knowledge  about  the  structure  of  the  problem  domain 
in  the  form  of  rewrite-rules.  For  the  clearblock-prohlem  we  provide  the  following 
information 
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;  Predicates  which  indicate  constructive  rewriting 
(defun  is-c-pred  (p-name) 

(equal  p-name  ^on) 

) 


;  rewrite  rule 

;  p  is  a  proposition  (predicate -name  arguments) 

;  c-eq  returns  an  association  list 
;  of  constant  -  constructive-expression  pairs 
;  (on  X  y)  ==  [x  =  (topof  y)] 

(defun  c-eq  (p) 

(list  (nth  1  p)  (list  ^topof  (nth  2  p)))  ;  assoc-list 

). 

A  proposition  on{Ki ,  K2)  gives  us  the  information  that  constant  Ki  is  on  the  top 
of  constant  i^2*  So,  we  use  all  on*predicates  in  a  linear  (sub-)  plan  to  generate 
bindings  of  constants  with  their  constructive  equivalent.  This  idea  corresponds 
to  Manna  and  Waldinger’s  use  of  the  /lat-axiom  [MW87]: 

if  not  Clear{w^  y) 

then  On{w,  hat{w^  y),  y)  for  all  states  w  and  blocks  y. 

For  the  clearhlock-pTohlem  we  obtain  the  bindings  ((A  =  topof(topof(C))) 
(B  —  topof ( C)))hy  applying  the  following  (here  only  informally  given)  algorithm 

•  Find  substitutions 

—  For  each  proposition  p  of  each  state  Do 

*  If  is-c-pred  (predicate-name  of  p) 

then  return  (c-eq  p)  (i.e.  the  binding  of  the  constant (s)  with  their 
constructive  equivalent) 

*  Union  cdl  found  bindings 

•  Apply  substitutions 

—  For  each  proposition  of  each  state  Do 

*  As  long  as  applying  the  substitutions  results  in  a  different  form  of  the 
arguments  of  the  proposition  Do 

replace  the  arguments  by  their  constructive  equivalence. 

‘Tind  substitutions”  gives  us  ((A  —  topof(B))  (B  —  topof(C)))]  and  “Ap¬ 
ply  substitutions”  rewrites  all  propositions  of  each  state  recursively.  That  is, 
topof  (B)  gets  rewritten  to  topo f  {topof  (C)) .  For  problems  more  complex  than 
clearblock  rewriting  is  done  for  each  linear  subplan  (see  chapter  3).  The  output 
generated  by  DPlan  is  given  in  figure  2.2.  Now  -  as  for  the  righthand  sides 
of  Summers’  straight-forward  programs  -  the  operations  are  represented  in  a 
constructive  way  and  -  what  we  will  need  for  constructing  the  boolean  condi¬ 
tions  -  the  states  itself  are  represented  over  a  constructive  data  type  also.  The 
remaining  constant  symbols  in  the  plan  can  now  be  interpreted  as  variables  (in 
correspondence  to  Summers’  abstract  forms  of  the  original  inputs). 
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Figure  2.2:  Result  of  constructively  rewriting  constants  for  clearblock 


Using  background-knowledge  -  as  Summers’  cpo  or  Manna  and  Waldinger’s 
axioms  —  is  an  important  technique  for  learning  cyclic  macro-operators  in  plan¬ 
ning.  While  background  knowledge  is  seldomly  regarded  in  classification  learn¬ 
ing  [Mit97]  it  is  central  (domain  theory)  in  explanation  based  learning  [MKKC86] 
and  also  common  practice  in  the  context  of  program  synthesis  from  examples 
(see  cf.  the  inductive  logic  programming  literature  [MDR94,  FYar]  and  the 
genetic  programming  literature  [Koz94])®. 

2.4  Determining  relevant  predicates 

In  the  next  step,  will  determine  the  minimal  set  of  relevant  predicates  of  each 
constructively  rewritten  state  of  the  plan.  This  corresponds  to  Summers’  in¬ 
troduction  of  boolean  expressions  as  the  lefthand-sides  of  his  straight-forward 
programs  -  and  ins  some  way  also  to  the  discrimination  predicates  used  in 
conditional  planning  [PS92,  BV97].  The  relevant  predicates  can  be  identified 
easily  from  the  information  obtained  during  planning:  Each  legal  planning  step 
consists  of  an  instantiated  operator  which  transforms  a  state  (the  state  newly 
constructed  by  backward-planning)  into  the  state  given  at  the  current  node. 
That  is,  the  predicate(s)  in  the  add-list  of  the  operator  must  match  with  pred- 
icate(s)  of  the  current  state.  These  predicates  are  the  one  which  are  fulfilled  in 
the  current  planning  step  and  therefore,  they  are  the  relevant  predicates. 

For  the  c/ear6/oc^-example  we  obtain:  (ct  C)  for  the  root-node  which  is 
generated  by  putt  able  (topof  C)  and  (ct  (topof  C))  for  the  second  node  which  is 
generated  by  putt  able  (topof (topof  C)).  The  leafs  of  a  plan  represent  states  for 
which  exists  no  predecessor  in  the  given  problem  domain.  For  leaf  nodes,  we 
regard  the  predicates  given  as  preconditions  of  the  last  applied  operator.  Thus, 
we  obtain  {(ct  (topof(topof  C))),  (on  (topof(topof(C))  (topof  C)))]  as  relevant 

^In  some  planning  systems,  cf.  PRODIGY  [VCP+95],  the  planner  can  be  provided  with 
additional  information  about  control  rules  to  make  planning  more  efficient  for  a  given  domain; 
similarly,  such  systems  could  be  extended  to  provide  information  about  the  data  structures 
of  the  planning  domain 
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Figure  2.3:  Binary  planning  tree  for  clearhlock 


predicates  for  the  last  state.  But  the  predicate  {on  (topof(topof(C))  (topof  C))) 
is  not  really  relevant.  It  plays  no  role  with  regard  to  a  general  strategy  for 
clearing  a  block.  There  are  at  least  two  heuristic  strategies  for  cleaning  the  leaf- 
node  predicates:  First,  we  could  assume  that,  if  up  to  now  only  the  operator 
putt  able  was  used  in  planning  this  would  also  be  the  case  for  larger  domains  and 
accordingly  replace  the  leaf-predicates  by  the  instantiated  add-list  of  putt  able. 
Secondly,  we  could  compare  the  predicates  used  so  far  and  only  accept  such 
predicates  of  the  preconditions  which  are  also  used  in  the  predecessor  nodes. 
But  we  will  see,  that  we  don^t  need  such  heuristics  because  the  leaf  states  are 
not  regarded  when  rewriting  the  plan  to  an  initial  program. 


2.5  Generating  a  binciry  tree 

Now  we  are  nearly  done  with  transforming  the  original  plan  tree  into  a  format 
suitable  for  generalization.  We  will  rewrite  the  plan  into  a  binary  structure. 
This  structure  is  similar  to  the  plans  proposed  by  [Wys87]  and  can  be  used  to 
generate  the  shortest  operator  sequences  for  a  set  of  input  states  (similar  to  a 
universal  plan)  by  means  of  an  interpreter  function. 

In  case  of  linear  plans,  generating  the  binary  tree  is  simple:  For  each  predi¬ 
cate  node  we  introduce  a  left  successor  s.  The  variable  s  represents  that  there  is 
a  situation  where  the  predicate  given  at  the  predecessor  node  is  true,  or,  more 
generally,  that  the  predicates  given  on  the  path  to  this  leaf  are  true.  The  right 
successor  is  the  next  predicate  node  with  the  arc  labelled  by  the  operation.  If 
there  is  no  further  operation  given,  we  terminate  the  path  with  an  asterix,  in¬ 
dicating,  that  for  this  case,  we  have  made  no  experience  during  planning.  The 
binary  tree  for  the  cl earblock-pr ohlem  is  given  in  figure  2.3. 

In  case  of  non-linear  plans,  we  try  to  unify  different  paths.  If  there  still 
remain  non-linear  subplans,  we  can  either  introduce  an  arbitrary  sequence  or 
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we  can  try  to  generalize  over  the  tree  structure  (see  chapter  4).  Note,  that 
for  unifying  different  paths  of  a  plan  it  is  crucial  that  constructive  rewriting 
has  been  performed  before:  when  searching  a  generalization,  it  is  meaningless 
to  unify  sub-paths  involving  identical  constants.  But  it  makes  sense  to  unify 
sub-paths  with  identical  constructive  expressions! 


2.6  Transforming  the  plan  tree  into  a  program 
term 

In  the  last  step,  we  rewrite  the  final  plan  into  the  syntax  required  for  initial 
programs  as  input  in  our  program  synthesis  system^.  To  obtain  the  initial 
program  for  cl  ear  block  given  above 

G  =  g{cleartop{x)jS,puttable{topof{x)j 

g{cleartop{topof{x))^s,puttable{topof{topof{x))^ 

fi)))) 


we  regard  each  predicate  node  as  a  boolean  condition,  its  left  successor  as  the 
then-case  and  its  right  successor  as  else-case.  Remember  that  the  remaining 
constants  in  the  rewritten  plan  are  regarded  as  variables. 

The  plan  can  be  read  as  “If  block  C  is  clear  then  return  the  current  situation, 
else,  put  the  top  of  block  C  on  the  table  in  a  situation  for  which  holds;  if  the 
top  of  block  C  is  clear  then  . . .”.  So,  if  an  arc  is  labelled  with  an  operator,  the 
subtree  becomes  the  additional  argument  of  the  operator.  If  the  subtree  of  an 
operator  does  not  contain  a  further  operator  and  if  there  is  an  asterix  in  this 
subtree,  we  terminate  rewriting  by  introducing  Q  (representing  the  undefined 
sequence  in  infinite  term  algebras  [Wys83,  CN78])  as  argument. 

Note,  that  the  clearblock  macro  not  only  generalizes  over  the  size  of  the 
blocksworld  domain  (number  of  blocks),  but  also  over  the  position  of  the  block 
which  is  to  be  cleared.  In  planning,  we  made  experience  with  clearing  the 
bottom  block  of  a  tower,  the  recursive  program  can  clear  an  arbitrary  block  x. 


^Remark:  this  step  is  not  implemented  yet.  Currently,  we  provide  “handmade”  initial  pro¬ 
grams  or  unfoldings  of  given  recursive  programs  to  our  synthesis  system.  Eckhard  Wiederhold 
now  is  implementing  the  transformation  as  part  of  a  student  project. 


Chapter  3 

DPlan  —  A  non-linear 
backward  planning  system 


It  may  be  a  little  unusual  to  propose  a  non-linear  backward  planning  algorithm 
when  the  state  of  the  art  is  graphplan  [BF97,  KNH97]  and  SAT-planning  [KS98]. 
But  from  the  perspective  of  learning  in  planning  efficient  plan  construction  is 
not  the  crucial  question.  From  a  cognitive  point  of  view  [SWar],  we  want  to 
model  how  experience  with  a  small  finite  problem  domain  can  be  generalized 
to  an  efficient  solution  strategy  for  a  complete  class  of  problems.  It  is  not  very 
plausible  to  assume  that  a  human  problem  solver  would  explore  a  logistics  prob¬ 
lem  with  105  action  steps  [Wel99b].  In  fact,  we  can  assume  that  it  is  possible 
to  extract  a  general  strategy  as  ‘‘try  to  load  as  many  objects  as  possible  in  the 
transporter  if  at  one  place”  from  very  small  problem  domains  (see  transporta¬ 
tion  example  in  section  4).  From  the  perspective  of  planning  we  propose  an 
automated  approach  to  generalize  finite  plans  into  cyclic  macro-operations.  For 
our  synthesis  method  described  shortly  in  section  2.1,  initial  programs  which 
correspond  to  the  third  expansion  of  a  hypothetical  recursive  program  scheme 
are  sufficient  [MS98].  So,  usually  it  is  enough  to  construct  plans  in  domains 
consisting  of  three  objects  of  each  given  type  and  clearly  for  domains  of  such 
restricted  size  efficiency  questions  are  not  crucial.  On  the  other  hand,  we  argue 
that  the  availability  of  iterative  macro-operations  can  lead  to  high  efficiency 
gains  when  the  planner  is  confronted  with  a  problem  of  a  already  known  class 
(cf.  Tower  of  Hanoi  with  3  discs  or  sorting  of  lists  with  three  elements)  but  of 
different  size  (n  discs,  n  list  elements).  Similar  arguments  as  well  as  calcula¬ 
tions  of  efficiency  gains  and  experimental  data  for  iterative  macro-operators  are 
presented  by  Shell  and  Carbonell  [SC89], 

In  this  chapter  we  will  present  the  planning  system  DPlan  and  discuss  it  in 
relation  to  current  planning  methodology.  The  transformation  of  the  originally 
output  of  DPlan  into  an  initial  program  is  discussed  in  chapter  4. 
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3.1  Semantics  for  D-plans 

The  planner  DPlan  produces  the  minimal  spanning  tree  of  a  finite  problem  space 
(for  an  overview  of  algorithms  for  calculating  spanning  trees  of  graphs  see  cf. 
[GH85,  Tar83]).  The  root  of  the  spanning  tree  are  all  problem  states  which 
are  subsumed  by  the  planning  goal.  The  minimal  spanning  tree  represents  the 
shortest  operation  sequences  for  all  problem  states.  Because  this  idea  is  similar 
to  the  Dijkstra- algorithm  for  calculating  all  shortest  paths  to  a  given  node  in  a 
graph  [Dij59],  we  call  our  planner  DPlan  and  the  resulting  plans  D~plans. 

3*1.1  Operators,  states,  and  operations 

First,  we  define  operators,  states,  operations  (actions)^. 

Definition  1  (Operator).  An  operator  is  a  4-tuple  consisting  of 

1.  a  name,  which  is  a  string, 

2.  the  precondition  which  is  a  conjunction  of  positive  literals, 

3.  an  operation  template  d,  which  is  the  operator  name  and  the  number 
and  names  of  its  arguments, 

4.  the  effect  (p  =>  atSt,aeSe,  with  (p  as  effect  condition  (limited  to  a  single 
positive  literal),  at  and  St  as  Add-  rsp.  Del-effects  if  (p  is  true  and  ag  and 
Sq  BiS  Add-  rsp.  Del-effects  if  p  is  false. 

When  ^  =  0,  at  =  0,  <Ji  =  0,  the  operator  is  “unconditioned”. 

Note,  that  in  contrast  to  the  definition  of  operators  in  other  planning  systems 
[PW92,  VCP'^95,  KNH97]  we  do  not  give  an  explicit  parameter  list.  Up  to 
now,  we  have  only  untyped  parameters.  Furthermore,  we  currently  allow  only 
a  single  effect  condition.  In  the  current  (first)  implementation  of  the  planner 
we  do  not  regard  negation,  disjunction  and  universal  quantifiers.  We  hope  to 
extend  our  approach  to  these  features. 

An  example  of  the  valid  operator  put  is  given  in  figure  3.1.  This  operator  has 
a  conditional  effect:  A  block  x  can  be  put  on  a  block  y  if  both  blocks  are  clear. 
Applying  this  operation  has  the  (primary)  effects  [FY95]  that  x  lies  on  y  and 
y  is  no  longer  free.  If  block  x  was  on  the  table,  these  are  the  only  effects;  but, 
if  X  was  lying  on  another  block  z,  there  are  the  additional  (side-)  effects  that  x 
does  no  longer  lay  on  z  and  z  is  clear.  Figure  4  shows  only  one  possibility  for 
representing  the  pwf-operator.  Another  possibility  is,  to  introduce  an  additional 
predicate  ontahle{x). 

Definition  2  (State).  A  state  is  a  set  of  positive  ground  literals  (ground 
atoms).  We  denote  a  positive  ground  literal  with  p  and  the  set  of  positive 
ground  literals  with  P,  As  usual,  a  state  5  G  2^  is  a  set  of  ground  atoms. 

Examples  for  states  are  {on(A,B),on(B,C),ct(A)},  {ct(A),  ct(B),  ct(C)}. 

^Terminology  and  sequence  of  our  definitions  follow  mostly  [KNH97]. 
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name:  put 

pre  :  {cleartop(x),  cleartop(y)} 
op  :  put(x,y) 
efF  :  if  on(x,z) 

then  ADD  {ct(z),  on(x,y)}  DEL  {on(x,z),  ct(y)} 
else  ADD  {on(x,y)}  DEL  {ct(y)} 

Figure  3.1:  The  operator  put 


name:  put 

pre  :  {cleartop(x),  cleartop(y),  for-all(z)  on(x,z)} 
op  :  put(x,y) 

efF  :  ADD  {on(x,y)}  DEL  {ct(y)} 
name:  put 

pre  :  {cleartop(x),  cleartop(y),  on(x,z)} 
op  :  put(x,y) 

efF  :  ADD  {ct(z),  on(x,y)}  DEL  {on(x,z),  ct(y)} 

Figure  3.2:  The  two  variants  of  the  put-operator 


Definition  3  (Operation/ Variants  of  an  operator).  An  operation  o  is  a 

ground  instance  of  an  operator.  That  is,  all  parameters  are  instantiated  with 
constants.  Additionally,  operations  have  unique  effects.  A  conditioned  operator 
(as  defined  in  def.  1)  results  in  two  operations:  (poU(f;  at,  St  and  (poU'-^(p]  Og, 

We  call  the  uninstantiated  forms  of  a  conditioned  operator  variants. 

We  use  the  term  operation  instead  of  the  often  used  term  action.  In  contrast 
to  other  planners  dealing  with  operators  with  conditioned  effects,  in  DPlan 
each  operation  has  a  unique  effect.  Instantiation  of  a  conditioned  operator  is 
performed  with  respect  to  its  variants.  The  variants  of  the  puf-operator  are  given 
in  figure  3.2,  Note,  that  while  DPlan  does  currently  not  allow  for  negation  and 
universal  quantifiers  in  specifying  operators  we  can  deal  with  such  expressions 
in  operator  variants^.  To  apply  ae,^e  we  have  to  make  sure  that  holds  for 
all  possible  instantiations. 

An  example  of  two  put-operations  derived  by  instantiating  the  put-operator 


^Of  course,  we  have  not  really  to  deal  with  universal  quantifiers  because  Var-ip (a?)  =  “i3p(a;). 
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oi  is  valid  for  {ct{A),ct{B),ct{C)}  in  a  blocksworld  of  three  blocks  A,  B,  C 
name:  put 

pre  :  {cleartop(B) ,  cleartop(C),  on(B,A)} 
op  ;  put(B,C) 

eff:  ADD  {on(B,C)}  DEL  {ct(C)} 

02  is  valid  for  {on{B,  A),  d{B),  ct{C)}  in  a  blocksworld  of  three  blocks  A,  B,  C 
name:  put 

pre  :  {cleartop(B) ,  cleartop(C),  on(B,A)} 
op  :  put(B,C) 

eff :  ADD  {ct(A),  oii(B,C)}  DEL  {on(B,A),  ct(C)} 

Figure  3.3:  Operations  derived  from  the  pu^-operator 

is  given  in  figure  3.3.  In  a  blocksworld  consisting  of  only  three  blocks  A,  B, 
and  C,  the  preconditions  constrain  the  possible  instantiations  of  z.  For  the 
examples  in  figure  3.3  we  have  a  =  {z  ^  A), 


3*1.2  Operation  applications,  admissible  states,  and  vari¬ 
able  bindings 

In  the  following  we  will  define  the  application  of  a  single  operation  and  of  opera¬ 
tion  sequences.  Furthermore,  we  will  describe  how  the  admissibility  of  calculated 
predecessor  states  is  checked,  when  operations  are  applied  backward  and  we  will 
describe  how  variable  bindings  are  determined  in  backward-planning.  Remem¬ 
ber,  that  our  planner  works  for  sets  of  initial  states^  i.e.  it  calculates  a  cpo  for 
all  states  of  a  finite  problem  domain. 

Definition  4  (Applying  an  operation).  We  denote  the  set  of  all  operators 
with  O.  Let  O  be  the  set  of  all  ground  instances  (operations)  of  O  and  Res  : 

X  O  — 2“^  be  a  function  from  states  and  operations  to  states. 

Application  of  a  single  operation  o  to  a  state  S  is  defined  as 

Rests  [SUA{S,o))\D{S,o)  ]ifpoQS 

undefined  ;  otherwise 

with  ^.(5,  o)  as  instantiated  ADD-list  of  o  and  Z)(S',  o)  as  instantiated  DEL-list. 

Definition  5  (Applying  an  operation-sequence).  The  function  Res  can  be 
expanded  to  Res  :  2^  x  O*  2^  with  O*  as  set  of  operation-sequences.  Ap- 
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plication  of  a  sequence  of  operations  can  be  defined  recursively  as 


R€s{S,  <  Oi,  . .  .On 


Res{S,  o)  ;  ?/  ||  <  oi . .  .On  >  ||  =  1 

Res{Res{S^<  >),On);  otherwise. 


The  forward  application  of  operation-sequences  is  used  in  plan  execution. 

Because  our  planner  is  working  backward,  starting  from  the  goal,  we  also 
have  to  define  backward  application  of  an  operation  and  backward  application 
of  a  set  of  operations. 


Definition  6  (Applying  an  operation  backward).  We  denote  the  set  of  all 
operators  with  O.  Let  O  be  the  set  of  all  ground  instances  (operations)  of  O 
and  Res  :  2^  x  O  — 2-^  be  a  function  from  states  and  operations  to  states. 
Application  of  a  single  operation  o  backward  to  a  state  S  is  defined  as 


(SU(D(So)U^c))\A(5,o) 

^  ^  (  undefined 


;  if  A{S,o)CS 
;  otherwise 


with  A(5,  o)  as  instantiated  ADD-list  of  o  and  D{S,  o)  as  instantiated  DEL-list. 

In  contrast  to  forward-planning,  we  have  to  check  whether  a  predecessor 
state  S'  calculated  by  Res~^{S,o)  is  admissible,  i.e.  if  it  is  a  legal  state  in  the 
planning  domain.  In  our  context,  a  state  is  admissible  if  it  is  consistent.  For 
example,  a  state  in  which  block  A  lies  on  block  B  and  block  B  is  cleartop  is  not 
admissible  because  it  contains  contradictory  propositions. 

Admissibility  can  be  checked  via  axioms  (see  [Wys87]  and  model-based  plan¬ 
ning  [CRT98]).  That  is,  S  =  {pi  . .  .pn)  G  2^  is  admissible  if  there  is  no  axiom 
where  from  some  pi^s  G  S  follows  -ipj  and  pj  G  S.  Another  possibility  to  check 
admissibility  is,  to  generate  a  candidate  predecessor  S'  by  backward  application 
of  operation  o  and  check  whether  Res{S' ,  o)  =  5  for  the  current  state  S. 

We  can  exploit  the  fact  that  our  planner  works  on  sets  of  initial  states  (i.e. 
all  states  of  a  given  finite  problem  domain).  Because  all  states  of  a  problem 
space  and  no  other  states  are  admissible  for  a  giving  planning  problem  we  can 
omit  the  introduction  of  axioms  and  defined  instead 


Definition  7  (Admissibility  of  states).  For  a  given  planning  problem  in  prob¬ 
lem  space  V  a  state  S  is  admissible  iff  5  G  P. 

We  illustrate  the  calculation  of  admissible  predecessor  states  by  backward 
application  of  an  operation,  again  with  the  familiar  blocksworld  example  (see 
figure  3.4).  For  the  clarity  of  the  example  we  assume  that  only  the  puf-operator 
given  in  figure  3.1  is  available  (for  a  complete  specification  of  the  itoii^er-problem 
we  have  to  provide  additionally  a  puttable-opevaioT,  see  section  2.2  and  below). 
We  obtain  four  instantiated  operators  consistent  with  the  current  state  S.  For 
the  three-block  world,  variable  .2:  can  only  be  instantiated  with  C  for  puti  and 
put 2  and  only  with  A  for  puts  and  put 4.  Only  the  application  of  puti  leads  to 
an  admissible  predecessor  state.  For  put 2  subtraction  of  the  Add-list  from  S 
can  not  be  performed  completely,  because  ct{C)  ^  S  and  there  is  no  state  in  T> 
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Set  of  all  admissible  states  of  the  three-block  blocksworld: 

Ptower  =  {  {on(A  B),  on(B  C),  ct(A)},  {on(B  C),  ct(A),  ct(B)},  {ct(A),  ct(B), 
ct(C)},  {on(B  A),  on(A  C),  ct(B)},  {on(C  A),  on(A  B),  ct(C)},  {on(A  C),  on(C  B), 
ct(A)},  {on(B  C),  oii(C  A),  ct(B)},  {on(C  B),  oii(B  A),  ct(C)},  {on(A  B),  ct(A), 
ct(C)},  {on(A  C),  ct(A),  ct(B)},  {on(B  A),  ct(B),  ct(C)},  {on(C  A),  ct(B),  ct(C)}, 
{on(C  B),  ct(A),  ct(C)},  } 

Current  state:  S  =  {on(A,  B),  on(B,  C),  ct{A)} 

Put-operations: 
name:  puti 

pre  :  {cleartop(A),  cleartop(B),  -  on(A,C)} 
op  :  put(A,B) 

eff  :  ADD  {on(A,B)}  DEL  {ct(B)} 
name:  put2 

I  pre  :  {cleartop(A),  cleartop(B),  on(A,C)} 
op  :  put(A,B) 

eff :  ADD  {ct(C),  on(A,B)}  DEL  {on(A,C),  ct(B)} 
name:  puts 

pre  :  {cleartop(B),  cleartop(C),  -■  on(B,A)} 
op  :  put(B,C) 

eff  ;  ADD  {on(B,C)}  DEL  {ct(C)} 
name:  puU 

pre  :  {cleartop(B),  cleartop(C),  on(B,A)} 
op  :  put(B,C) 

eff :  ADD  {ct(A),  on(B,C)}  DEL  {on(B,A),  ct(C)}. 

CcJculating  predecessors: 

Res-\S,puti)  =  {on{B,C),ct{A),ct{B)}^ 

Res-\S,puh)  =  {on{B,C),ct{A),on[A,C),ct{B)}* 

R£s~'^{S,puh)  =  {on(A,B),ct{A),ct{B),ct(C)}  * 

_ Res~\S,put4)  =  {on(A,B),ct(B},on{B,A),ct(C)}  * . 


Figure  3.4:  Calculating  predecessor  states  and  selecting  admissible  predecessors 
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where  two  blocks  are  lying  on  C  simultaneously.  There  are  also  no  state  in  V 
where  A  lies  on  B  and  B  is  cleartop  (or  where  A  lies  on  B  and  B  simultaneously 
on  A). 

The  crucial  (and  most  time-consuming)  aspect  in  backward-planning  is  to 
calculate  the  variable  bindings  [BW94]  which  are  used  to  instantiate  operators 
and  thereby  to  generate  predecessor  states.  As  for  determining  admissible  states, 
we  can  again  rely  on  the  predefined  set  of  states  V. 

Definition  8  (Determination  of  variable  bindings).  Let  S  be  the  current 
state.  For  each  variant  d  of  an  operator  with  precondition  (po  (which  can  either 
be  (po  0  (p  or  U  -^(p)  and  effect  aS  (which  can  either  be  atSt  or  aeSe)  we 
determine  the  set  of  legal  variable  bindings  in  the  following  way: 

L  i;=:0 

2.  Find  all  substitutions  ai  with  C  S. 

Set  E  to  E  =  {cTj  I  i  =  1 . . .  n}. 

3.  For  each  ctj  find  all  compatible  substitutions  p  with  (poaip  Q  S'  with 
S'  £V, 

Set  each  ai  to  <Tip, 

The  composition  of  substitutions  a  and  p  is  compatible  iff  for  {v  ^t)  E  <r  there 
exists  no  [v'  i—i')Ep  with  v  =  v'  and  t  ^  t',  where  t^t'  are  constants. 

We  will  show  below  that  by  using  look-ups  in  the  set  of  states  V  determining 
variable  bindings  can  be  performed  much  more  efficient  than  in  usual  backward¬ 
planning  approaches. 

Finally,  we  can  define  the  application  of  “parallel  operations”.  That  is,  we 
describe  how  the  complete  set  of  legal  predecessors  of  a  state  is  calculated. 

Definition  9  (Applying  a  set  of  operations  backward).  Let  ReSp^  be  a 
function  from  states  and  sets  of  operations  to  sets  of  states  Res^^  :  2^  x  2^  ^ 

2^^  and  V  the  set  of  all  states  of  a  given  finite  problem  domain. 

The  backward  application  of  a  set  of  operations  {oi . .  .o„}  results  in  a  set 
of  admissible  predecessor  states  {5i . .  .5m}  with  m  <  n  and  is  defined  in  the 
following  way: 

ReSp\S,{o})  =  Res-\S,  o) 

(  {Res-^{S,oi)}  U  Res-^{S,{oi+i . .  .On}) 

Res~^{S,  {oi . . .  o„})  =  <  with  i  =  1 . .  ,n  ;  if  ReSp^{S,  ol)  £  V 

[  ReSp^  (5,  {oi^i  ...  On})  with  2  =  1 . . .  n;  otherwise. 

For  the  state  {on(B,C),  ct(B),  ct(A)}  the  application  of  a  set  of  put  operations 
results  in  the  admissible  predecessors  {  {ct(A)j  ct(C)),  {on(B,A),  ct(B)j 

ct(C)}}.  Of  course,  when  planning  in  the  blocksworld  domain  we  also  regard 
applications  of  putt a6/ e-operations. 
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3.1.3  Implementation  details 

In  our  implementation,  operators  are  defined  in  the  following  way: 

(setq  rules  ’( 

(rule  puttable 

(if  (  ct  (>  x)  )  (  on  (<  x)  (>  y)  )  ) 

(then  (puttable  (<  x))) 

(conq  (  add  ((ct  (<  y))) 

del  ((on  (<  x)  (<  y)))  )  ) 

) 

(rule  put 

(if  (  ct  (>  x)  )  (  ct  (>  y)  )  ) 

(then  (put  (<  x)  (<  y)  )  ) 

(conq  (cond  ((on  (<  x)  (>  z)) 

(add  ((on  (<  x)  (<  y))  (ct  (<  z))) 
del  ((ct  (<  y))  (on  (<  x)  (<  z))))) 

(  T  (  add  ((on  (<  x)  (<  y))) 

del  ((ct  «  y)))  )  )  )  )  ) 

)) 

The  representation  of  variables  as  (>  x)  or  (<  y)  is  different  from  the  often 
used  form  ?x.  We  were  inspired  by  Winston’s  definition  of  match-select-apply 
cycles  for  production  systems  [WH89].  The  symbol  >  indicates  that  a  variable 
has  to  be  instantiated  in  accordance  to  the  current  state;  the  symbol  <  that 
a  variable  has  be  instantiated  in  accordance  with  the  current  set  of  bindings. 
For  backward-application  of  operations  the  semantics  of  <  rsp.  >  is  switched. 
In  the  next  implementation  we  want  to  replace  >  /  <  by  the  usual  ?  notation. 
This  can  be  realized  with  only  a  slight  modification  in  the  algorithm:  If  there 
is  a  variable:  first  look  in  the  list  of  current  bindings,  if  the  variable  is  bound, 
take  this  value,  otherwise,  obtain  the  binding  from  the  current  state. 

We  already  remarked  above  that  we  hope  to  expand  our  first  implementation 
of  DPlan  to  full  ADL-syntax  [Ped89].  Furthermore,  we  want  to  introduce  the 
possibility  of  defining  data  types  for  variables  and  we  want  to  allow  for  external 
function  calls  in  operations  similar  to  PRODIGY  [BCE+92].  These  extensions 
are  crucial  if  we  want  to  use  our  planner  for  calculating  initial  programs  for 
list  and  nnmber  problems  which  are  nsually  regarded  in  program  synthesis  (see 
chapter  4  and  appendix  E). 


3.2  The  algorithm  DPlan 

After  defining  the  backward  application  of  sets  of  operations  to  a  current  state 
thereby  generating  the  set  of  all  admissible  predecessor  states  we  are  ready  to 
define  the  planning  problem. 

Definition  10  (Planning  problem).  A  planning  problem  P(0,T>,^)  is  a 
3-tuple  where  O  is  the  set  of  operators,  V  a  set  of  initial  states  and  G  a  set  of 
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planning  goals.  All  states  S  EV  and  G  are  sets  of  positive  ground  literals.  In 
the  context  of  our  work,  V  is  the  set  of  all  states  of  a  finite  problem  domain. 

Planning  goals  are  currently  restricted  to  conjunctions  of  (possibly  interdepen¬ 
dent)  subgoals  (cf.  {on(A,B)^  on(B,C)}).  We  are  planning  to  extend  goal 
definitions  to  negation,  disjunction  and  universal  quantification  in  the  future. 

Let  C  be  the  set  of  all  constants  (i.e.  arguments  of  predicates  P  E  S) 
occurring  in  V,  The  set  of  operations  O  is  the  set  of  all  possible  ground  instances 
of  O  with  respect  to  C. 

Definition  11  (Minimal  spanning  tree  ofV)*  ^  The  minimal  spanning  tree 
of  a  planning  problem  0  =  {V^E)  with  the  set  of  nodes  I/,  the  set  of  edges 
E  CV  X  Vf  and  two  labeling  functions  /?v  :  V  V,  /3e  •  E  O  is  a,  tree  with 
arbitrary  branching  factor,  i.e. 

{A  {empty  tree) 

vEV  {leaf) 

v{0i  . .  .Ot)  with  0  <b  <  \\V\\ 

for  which  the  following  conditions  hold: 

(With  0/  we  denote  the  tree  from  the  root  to  level  I  and  with  Ni  we  denote  all 
nodes  in  0; .) 

1.  The  root  Q{NjE)o  is 

#  ±  if  there  is  no  state  S  E  V  for  which  G  C  S  holds  (the  planning 
problem  is  not  solvable),  or 

•  V  with  /3y{v)=SifSET>  and  G  C  S',  and  if  there  is  no  other  state 
S'  EV  with  G  C  S'  (there  is  exactly  on  state  which  fulfills  the  goal) , 
or 

♦  (root  node  and  first  level  of  0)  v{wi  ...Wn)  with  l3v{v)  =  G  and 
P{wi)  =  S  if  S  E  T>  and  G  C  S  and  /3E{v,Wi)  =  e  for  all  Wi, 
2  <  «  <  ll'Z^II  (there  is  more  than  one  state  fulfilling  the  goal). 

2.  For  each  leaf  node  v  of  0/  the  children  are  all  w  E  Res^^  {/3v  {'i’) ,  O)  with 
w  is  admissible  (i.e.  w  E  T>)  and  w  ^  Ni  .  For  each  edge  (v,  w)  we  have 
the  label  I3e{v,w)  =  o  with  Res~^{/3v{v),o)  =  w. 

For  the  formal  background  and  a  variety  of  algorithms  for  spanning  trees,  see 
cf.  [GH85,  Tar83]. 

Definition  11  entails  the  planning  algorithm  (which  we  already  described 
informally  in  chapter  2)  given  in  table  3.1"^.  We  will  discuss  its  soundness, 

^Remark:  Def.  11  and  the  presentation  of  the  algorithm  are  a  bit  overloaded.  I  will 
formulate  both  “more  elegantly”  soon. 

^The  initialization  and  the  tree  expansion  are  not  easy  to  read,  because  I  discriminate  be¬ 
tween  nodes  v,  w  and  node  labels  I3y{v)  which  are  corresponding  to  states.  When  simplifying 
def.  11  the  algorithm  can  be  simplified  accordingly.  Note,  that  the  planning  algorithm  can 
be  implemented  more  efficient  when  integrating  the  calculation  of  variable  bindings  and  state 
candidates  in  one  loop,  see  efficiency  of  DPlan  below. 
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completeness  and  optimality  in  section  3.4  and  we  will  give  further  examples 
below  and  in  section  4.  Information  about  the  current  implementation  of  DPlan 
is  given  in  appendix  A. 

Note,  that  in  the  implementation  of  DPlan  we  generate  a  graphical  output 
of  0  where  the  operations  are  not  given  as  edge  labels  but  as  intermediate  nodes 
between  nodes  representing  problem  states. 

Illustration:  a  blocksworld  example 

To  generate  a  plan  for  building  a  tower  of  blocks  A,  B,C  in  a.  three- 
block  world,  we  present  DPlan  with  the  following  problem  specification: 

(setq  states  ^ ( 


((on 

a 

b) 

(on 

b 

c) 

(ct  a)) 

((on 

b 

c) 

(ct 

a) 

(ct  b)) 

((ct 

a)  (ct  b; 

)  (ct 

c)) 

((on 

b 

a) 

(on 

a 

c) 

(ct  b)) 

((on 

c 

a) 

(on 

a 

b) 

(ct  c)) 

((on 

a 

c) 

(on 

c 

b) 

(ct  a)) 

((on 

b 

c) 

(on 

c 

a) 

(ct  b)) 

((on 

c 

b) 

(on 

b 

a) 

(ct  c)) 

((on 

a 

b) 

(ct 

a) 

(ct  c)) 

((on 

a 

c) 

(ct 

a) 

(ct  b)) 

((on 

b 

a) 

(ct 

b) 

(ct  c)) 

((on 

c 

a) 

(ct 

b) 

(ct  c)) 

((on 

c 

b) 

(ct 

a) 

(ct  c)) 

)) 

(setq 

goal 

^  (  (on 

a  b)  (on 

b  c 

)  ) 

) 


The  ordering  of  states  and  of  goal  propositions  is  arbitrary.  The  op¬ 
erator  definitions  for  put  and  putt  able  were  already  presented  in  section 
3.1.3. 

Planning  starts  with  checking  whether  there  are  states  fulfilling  the 
goal:  state  ((on  a  b)  (on  b  c)  (ct  a))  fulfills  the  goal  and  no  other 
state,  therefore  it  gets  the  root  of  the  plan  and  is  deleted  from  the  list  of 
current  states  (see  “initialization”  in  table  3.1). 

In  the  next  step  (first  node  expansion,  see  table  3.1),  DPlan  gener¬ 
ates  aU  such  instantiations  of  all  operator  variants  which  ADD-hst  lit¬ 
erals  match  with  a  literal  of  the  current  state.  That  is,  candidates  are 
(puttable  x)  with  y  =  A  realizing  (ct  A),  (put  A  B)  with  and  with¬ 
out  (on  A  z)  realizing  (on  A  B)  and  (put  B  C)  with  and  without  (on 
B  z)  realizing  (on  B  C).  Each  operation  candidate  is  applied  backwards 
(adding  literals  from  the  DEL-list  and  preconditions  and  deleting  literals 
from  the  ADD-list  from  the  current  state).  The  calculation  of  state  can¬ 
didates,  selection  of  admissible  candidates,  and  determination  of  variable 
bindings  was  already  illustrated  in  section  3.1.2.  In  figure  3.4  we  demon¬ 
strated  for  the  put  operator,  that  only  one  of  the  four  candidates  results  in 
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Table  3.1:  The  planning  algorithm  DPlan 
input  :  a  planning  problem  T{0^  X>,  G) 

output  :  the  minimal  spanning  tree  0  for  V  or  1, 

initialization;  A  =  2>,  the  set  of  current  states 


e:={ 


(l)i-;  if  there  exists  no  state  S  £  V  with  G  C  S 

{2)v  with  l3v{v)  =  S  £V  and  G  C  S;  if  there  exists  no  other  state 

S'  ev  with  Gcs' 

(3)i;(wi  . . .  Wn)  with  l3v{v)  =  G  and  pv{wi)  =  Si  \fSi  €  V  and  G  C  5^, 
with/3E{Gj  Si)  =  e,  2  =  1 . . .  n,  2  <  n  <  ||P||;  otherwise 


for  (2)  and  (3)  A  is  reduced:  A  :=  A  {l3v{v)}  for  (2)  and  A  :=  A  {l3v{wi)}ywi 
for  93). 

•  Terminal  condition;  A  =  0  (success)  or  for  which  def.  11.3  holds  (failure) 

•  Tree  expansion;  (1) 

If  0  =  ± 

Then  return  bottom 

Else  For  all  leafs  Vi  of  0  =  ^(t;i . . .  Vn)  do 

1.  Calculate  Q  =  {w\/3v(w)  G  ReSp^(/3v{v)yO)  G  A} 

2.  A  :=:  A\{l3v(w)}forallw  ^  Q 

3.  Expand  0: 

Replace  leaf  Vi  by  . . .  «^m)  with  fisivi^Wj)  =  o  with 

Resp^{l3v{vi),0)  =  {wi  ...Wm}- 

endfor 

•  Node  expansion:  (2) 

For  the  current  state  S  =  l3v{v)  and  for  each  variant  6  of  an  operator  in  O  with 
d  =  ipojOaS 

-  Determination  of  all  sets  of  legal  bindings  (2a) 

1.  S  =  0 

2.  Find  all  substitutions  Ci  with  C  S 

3.  E  :=  {(Ti  I  2  =  1 . . .  fc} 

4.  For  each  Ci  find  all  compatible  substitutions  p  with  C  S'  with 

S'  €V. 

5.  Set  each  (Ti  G  E  to  <Ti  :=  (Tip, 

—  Construct  all  operation  candidates  O  =  {o  |  daf] 

~  Calculate  all  state  candidates 

C=E..-U<i  {SU{D(S,o)yj^o))\A{S,o)  ■,ifA{S,o)CS 

^  \  undefined  and  O  :=  O  \  {o}  ;  otherwise 


with  A{S^o)  as  instantiated  ADD-list  of  o  and  £>(5,  o)  as  instantiated 
DEL-list. 

-  For  all  c  G  C  with  c^AdoC:=C\c 

—  Return  all  pairs  (o,5')  with  S'  =  Res~~^(S,o)  G  C 
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Figure  3.5:  Result  of  DPlan  for  the  tower  problem 


an  admissible  predecessor  —  (put  A  B)  where  A  was  not  lying  on  another 
block. 

We  give  an  additional  illustration  for  putt  able:  First  we  find  the  legal 
bindings  x  =  B  and  x  =  C  (see  2a  in  table  3.1)  resulting  in  the  operation 
candidates: 

name:  puttablei 
pre:  {cleartop(B)} 
op:  puttable(B) 

eff:  ADD  {cleartop(A)}  DEL  {on(B,A)} 


name:  puttable2 
pre:  {cleartop(C)} 
op:  puttable(C) 

eff:  ADD  {chart op ( A J}  DEL  {on(CfA)} 

with  the  not  admissible  predecessors 

Res-^  {S,puttable\)  -  {on{A,B),on{B ,C),on(B,A),ct(B)} 

Res'''^  {S,puitable2)  =  {ori{A,B),on{B,C),ori\c,  A.)yCt{C)}. 

That  isj  the  expansion  of  the  root  results  in  the  application  of  only  one 
operation  (put  A  B)  for  the  admissible  predecessor  state  ((on  B  C)  (ct 
A)  (ct  B)).  This  state  is  deleted  from  the  set  of  current  states.  Plan 
expansion  proceeds  for  this  single  leaf  which  has  two  legal  predecessors 
and  so  on.  The  final  plan  is  given  in  figure  3.5.  Note,  that  for  initial  states 
corresponding  to  towers  whose  blocks  are  sorted  in  reverse  to  the  desired 
goal,  the  blocks  have  not  to  be  unstacked  completely.  Instead  the  side- 
effect  of  put  -  clearing  a  block  z  if  x  lies  on  a  block  and  not  on  the  table  ~ 
can  be  exploited  for  an  optimal  plan.  While  this  plan  is  more  intelligent 
than  simply  first  putting  all  blocks  on  the  table  and  then  constructing  the 
desired  tower  it  comphcates  macro  synthesis:  reverse  sorted  towers  has 
to  be  considered  as  exception  and  treated  differently  than  all  other  input 
states. 
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Table  3.2:  Plan  execution  as  depth-first  search 

•  Let  I  be  the  input  state  (initial  state)  and  L  =  ((©o))  the  list  of  paths 
(00  is  the  root  of  the  plan). 

•  Until  L  =  (  )  or  H ead(H ead[L))  =  I  Do 

-  Remove  La  =  Head{L)  from  L. 

—  For  Head{La)  find  all  immediate  successors  5^ , «  =  0  . . .  n  in  0  with 
edges  {Head{La),Si)  =  Oi. 

-  Create  new  lists  {Si,  Oi^  Tail  {La)). 

~  Insert  the  lists  in  front  of  Tail{L). 

Then  announce  failure  (/  is  no  admissible  state  in  0) 

Else  Res{I ,Tail{H ead{L)))  (apply  operation  sequence) 


3.3  Plan  execution 

The  output  of  DPlan  represents  operation  sequences  for  all  possible  states  of 
a  given  domain.  For  example,  the  Dplan  for  the  tower  problem  (see  figure 
3.5)  contains  the  plans  for  building  a  tower  A,  B,  C  from  initial  state  on(C,B), 
on(B,A),  ct(C)  as  left  most  path,  from  initial  state  on(B,C),  ct(A),  ct(B)  as 
partial  path  and  so  on. 

Note  that  representing  action  sequences  for  each  possible  state  differs  from 
the  state-action  tables  used  in  universal  planning  [Sch87,  Sch95].  We  can  calcu¬ 
late  action  sequences  because  DPlan  is  restricted  to  deterministic  worlds.  That 
is,  state  changes  can  only  occur  as  result  to  an  operation  application  and  there 
are  no  possible  influences  from  other  agents  or  the  environment.  Thus,  it  cannot 
be  the  case,  that  one  of  the  planning  objects  is  suddenly  missing  or  is  not  at  the 
expected  position.  With  this  restrictions,  plan  execution  can  be  performed  a 
simple  tree-search  algorithm.  Both  depth-first  (see  table  3.2)  and  breadth-first 
(see  table  3.3)  search  are  possible  (we  use  a  similar  notation  as  [Win92]). 

For  the  abstract  tree  given  in  figure  3.6,  the  depth-first  algorithm  generates 
the  following  sequence: 

I  =  g,  L  =  ((a)) 

L  =  ((b  ol)  (c  o2)  (d  o3)) 

L  =  ((e  o4  ol)  (f  o5  ol)  (c  o2)  (d  o3)) 

L  =  ((f  o5  ol)  (c  o2)  (d  o3)) 

L  =  ((c  o2)  (d  o3)) 

L  =  ((g  o6  o2)  (d  o3)) 

->  apply  <o6,  o2>. 

The  corresponding  sequence  for  the  breadth-first  algorithm  is: 
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Figure  3.6:  Abstract  minimal  spanning  tree 


Table  3.3:  Plan  execution  as  breath-first  search 

•  Let  I  be  the  input  state  (initial  state)  and  L  =  ((©o))  the  list  of  paths 
(00  is  the  root  of  the  plan) . 

♦  Until  L  =  (  )  or  Head{Head[L))  —  I  Do 

-  Remove  La  =  Head{L)  from  L. 

-  For  Head[La)  find  all  immediate  successors  f  =  0  . . .  n  in  0  with 
edges  {Head{La),Si)  =  Of. 

-  Create  new  lists  {Si,Oi,Tail{La))^ 

—  Insert  the  lists  at  the  end  of  Tail[L). 

Then  announce  failure  (/  is  no  admissible  state  in  0) 

Else  Res{I ail{H ead{L)))  (apply  operation  sequence) 
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I  =  g,  L  =  ((a)) 

L  =  ((b  ol)  (c  o2)  (d  o3)) 

L  =  ((c  o2)  (d  o3)  (e  o4  ol)  (f  o5  ol)) 

L  =  ((d  o3)  (e  o4  ol)  (f  o5  ol)  (g  o6  o2)) 

L  =  ((e  o4  ol)  (f  o5  ol)  (g  o6  o2)) 

L  =  ((f  o5  ol)  (g  o6  o2)) 

L  =  ((g  o6  o2)) 

->  apply  <o6,  o2>. 

Alternatively,  we  could  decompose  the  plan  tree  and  save  a  state/action- 
sequence  table  (as  (a  nil),  b  ol,  . . ,  (e  o4-ol),  . . .).  Searching  the  table  has 
the  same  complexity  as  search  in  the  tree  (if  we  do  not  introduce  a  efficient 
hashtable  representation  mechanism). 


3.4  Termination,  Soundness,  Completeness,  Op¬ 
timality,  Efficiency 

As  usual,  we  can  give  no  guarantee  that  the  given  specification  of  a  problem 
(i.e.  goal,  operators  and  set  of  states)  are  sound  and  complete.  The  following 
theorems  and  proofs  address  the  characteristics  of  DPlan  with  respect  to  a  given 
problem  specification. 

Theorem  1  (Termination).  For  given  planning  problem  V {0,1),  Q  DPlan  ter¬ 
minates  either  successful  and  returns  the  minimal  spanning  tree  0  ofV  or  it 
terminates  unsuccessful  and  returns  X. 

Proof: 

(1)  First  we  proof  that  DPlan  detects  the  unsolvability  of  a  planning 
problem  (termination  with  X).  There  are  two  cases  of  unsolvability: 

•  Total  unsolvability:  There  is  no  state  S  GV  with  G  C  S. 

•  Partial  unsolvability:  The  set  of  states  A  C  T>  which  are  not 
yet  inserted  in  0  is  not  empty  but  there  exists  no  current  leaf 

in  0  for  which  Res~^{v,  o)  returns  an  admissible  state  in  A. 

Total  unsolvability  is  detected  before  the  entrance  in  the  planning 
loop  (see  table  1,  tree  expansion)  and  DPlan  terminates  immediately. 

In  the  case  of  partial  unsolvability  we  have  a  current  tree  0^  = 
v{vi . . ,  Vn)  with  leafs  Vi,i  =  1 . .  .n.  If  Res^^{vi,  O)  =  0  then  DPlan 
tries  to  expand  vi^i.  If  ReSp^{vi,0)  —  0  for  all  Vi  than  DPLan 
terminates  with  0^  =  Thetat+n  and  returns  X. 

(2)  DPlan  terminates  successfully  if  all  states  in  T>  are  regarded 
in  0,  i.e.  A  =  0  (this  is  a  termination  condition  for  the  planning 
loop,  see  table  1)^.  q.e.d. 

^Because  partial  solutions  might  also  be  of  interest,  in  the  implemented  version  DPlan 
returns  the  partial  spanning  tree  but  without  announcing  success. 
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Theorem  2  (Soundness).  ®  If  DPlan  returns  a  minimal  spanning  tree  0  for 
a  given  planning  problem  P{0,'D,Q  then  each  path  from  an  arbitrary  node  S  G 
Theta  to  the  root  is  a  operation  sequence  Q  =<  o,  >  with  0  <  i  <  n  with 
G  C  Res{Si,<  Oi  >),  Le.  Q  is  a  solution. 

Proof: 

We  prove  soundness  by  induction  over  the  length  of  operation  se¬ 
quences  relying  on  definitions  11,9  and  5. 

[base  case]  Q  =<> 

We  have  to  show  that  Q  =<>  holds  exactly  for  such  states  S  with 
G  C  S.  In  definition  11  no  operations  are  introduced  only  if  V  is 
totally  unsolvable  (but  then  0  =  J.),  or,  if  there  exists  a  unique 
state  S  with  G  C  S',  or  if  there  exists  a  set  of  states  S  with  G  C  S. 

So  for  all  cases  were  0  ^  X,  the  empty  sequence  of  operations  is 
only  valid  for  the  root  of  0  as  defined  in  definition  11.1. 

[induction  step]  Q  =<  oi . .  .o„  > 

We  regard  an  arbitrary  path  from  a  node  5  G  0  to  the  root  of  0  with 
edge  labels  <  o„  . .  .oi  >.  We  assume  that  the  induction  hypothesis 
holds  for  this  path,  i.e.,  G  C  Res{S,  <  o„  . . .  oi  >.  During  planning, 
we  calculate  all  admissible  predecessors  of  S  not  yet  regarded  in  0. 
Applying  Res~^{S,  O)  can  have  the  following  results: 

1.  Res;\S,0)  =  ID 

2.  Res~^{S,0)  =  S'  with  o„+i  =  (S',S) 

3.  i?es-i(5,0)  =  S'i...S^  with  {o„+i;  |  o„+i  =  (5^,5)VSi  G 
Res;^S,0)}. 

For  case  (1)  we  do  not  introduce  a  new  node  (and  thereby  also  no 
new  operation),  that  is  G  C  Res{S,<  o„  ...oi  >  still  holds.  For 
case  (2)  we  have  an  admissible  predecessor  of  S  in  accordance  to 
definition  9.  That  is,  ResjS',  o„+i)  =  S  and  therefore,  in  accordance 
to  definition  5,  G  C  Res(S',<  o„+i,  o„  . .  .oi  >).  For  case  (3)  we 
have  a  set  of  admissible  predecessors  of  S  in  accordance  to  definition 
9.  That  is,  o„+i,)  =  S  for  all  S'j  G  Res-^{S,0).  Therefore, 

in  accordance  to  definition  5,  G  C  Res{S'i ,  <  o„+i , ,  o„  . . .  oi  >)  holds 
for  all  Si.  q.e.d. 

Theorem  3  (Completeness).  Completeness  follows  from  soundness  and  ter- 
mination. 

Theorem  4  (Optimality).  For  a  planning  problem  V{0,V,g  the  resulting 
plan  0  is  a  minimal  spanning  tree  for  problem  space  V  and  it  represents  the  min¬ 
imal  sequence  of  operations  for  transforming  each  state  S  eV  into  a  state  fulfill¬ 
ing  the  goal.  That  is,  G  C  Res{S,  <  oi . . .  o„  >)V5  G  V  and  -i3  <  oi  . . .  o'm  > 
with  G  C  Res[S,  <  oi . .  .o'^  >)  with  m  <.  n.  In  other  words,  Theta  is  a  span¬ 
ning  tree  with  minimal  depth  for  V. 

^Remark:  The  proof  could  be  more  compact  if  I  introduce  a  definition  for  the  relation 
between  paths  in  ©  and  forward-application  of  operation  sequences. 
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Proof: 

We  can  proof  this  theorem  by  structural  induction  over  the  levels  of 
0. 

[base  case:]  0  =  0o 

We  have  to  show,  that  the  root  0o  of  0  represents  all  states  S 
which  already  fulfill  G. 

This  follows  immediately  from  definition  11:  All  states  S  E  VwiihG  C 
S  are  regarded  when  introducing  the  root  of  0.  If  there  is  more  than 
one  state  fulfilling  G  C  5,  a  “dummy”-root  is  introduced  with  these 
states  as  immediate  successor  nodes  and  empty  edge  labels. 

[induction  step:]  0  =  0/ 

We  regard  a  partial  spanning  tree,  expanded  to  depth  /  covering  a 
subsets  of  states  from  V  for  which  the  induction  hypothesis  holds. 

That  is,  for  all  leafs  5  E  0/  holds  S  E  T>  and  G  C  Res{S,  < 
oi  . . .  >)  and  -»3  <  . . .  dm  >  with  G  C  Res{S,  <  >) 

with  m  <  n.  The  expansion  of  the  nodes  S  follows  definition 
11.  That  is,  an  admissible  predecessor  state  S'  =  Res~^{S,0)  is 
only  then  introduced  into  0  if  5^  ^  0/,  otherwise  S'  is  a  node  in 
0;.,  Ar  <  /  +  1  with  G  C  Res{S,  <  oi  , ,  .Ok  >).  q.e.d. 

Theorem  5  (Efficiency).  DPlan  calculates  the  minimal  spanning  tree  0  with 
a  worst  case  effort  of  0(n^y. 

Proof:® 

The  expansion  of  0  (outer  loop,  (1)  in  table  1)  needs  (n  —  1)  cycles 
in  the  worst  case.  That  is,  each  node  has  exactly  one  predecessor. 

So,  in  each  cycle  only  one  node  is  removed  from  A. 

The  expansion  of  a  node  in  0  (inner  loop,  (2)  in  table  1)  needs 
in  the  worst  case  ||A||  cycles.  That  is,  for  calculating  all  sets  of 
possible  bindings  and  determining  the  admissibility  of  predecessors, 
each  state  which  still  has  to  be  regarded  has  to  be  checked. 

So,  for  \\V\\  =  n  we  have  in  the  worst  case  n-f-n— 1  +  ?^^2...1  = 

^  ^  ri?  steps,  q.e.d. 

We  want  to  conclude  this  section  with  some  remarks  regarding  elficiency  of 
DPlan.  The  complexity  0{n^)  is  not  too  bad  for  a  backward  planning  system 
which  additionally  regards  not  a  single,  but  a  complete  set  of  initial  states. 
The  efficiency  gains  in  comparison  with  other  backward  planners  (cf.  TOPI, 
[BW94])  is  mainly  due  to  the  fact  that  we  can  use  look-ups  to  a  (stepwise  get¬ 
ting  smaller)  set  of  states  for  calculating  variable  bindings  (which  has  complexity 
1 1 /|  I II  ^11  with  ||J||  as  number  of  literals  in  the  initial  state  and  ||G||  as  number 

^This  holds  for  an  optimized  version  of  the  algorithm  given  in  table  1  and  the  current 
implementation  where  all  calculations  performed  after  step  4  of  calculating  the  bindings  (in 
expand  node)  are  also  performed  in  step  4. 

® Remark:  currently  very  quick  and  dirty!  Especially  the  aspect  of  calculating  sets  of 
bindings  should  be  checked  again. 
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of  goal  literals  for  TOPI,  that  is,  is  NP-hard  in  the  worst  case®).  Further¬ 
more,  by  calculating  a  minimal  spanning  tree,  DPlan  works  completely  without 
backtracking.  Backtracking  can  in  the  worst  case  result  in  calculating  the  com¬ 
plete  search  tree.  For  a  problem  domain  with  n  states  and  m  operations,  the 
worst  case  complexity  is  n*".  Finally,  generation  of  optimal  (i.e.  shortest)  plans 
can  for  ADL-planners  only  be  guaranteed  by  using  a  breadth-first  strategy  (for 
example,  IPP  [KNH97]  cannot  guarantee  optimality).  DPlan  uses  a  kind  of  op¬ 
timized  breadth-first  search  -  where  the  number  of  state  candidates  shrinks  at 
each  level.  Finally,  we  propose  that  our  strategy  of  storing  all  (not  yet  regarded) 
alternative  operations  and  their  outcomes,  could  be  useful  also  in  the  context  of 
other  planning  systems:  Usually,  for  each  planning  step  all  alternative  actions 
are  calculated,  but  after  selecting  one  action  the  other  ones  are  forgotten.  Thus, 
in  case  of  backtracking,  all  options  have  to  be  calculated  again. 

One  might  argue  that  we  load  the  greater  part  of  effort  on  the  users  of 
DPlan  who  have  to  specify  all  states  of  a  problem.  We  have  some  arguments 
against  this  proposition:  (1)  Remember,  that  our  main  goal  is  providing  initial 
programs  for  program  synthesis  or  -  taking  a  planning  perspective  -  general¬ 
izing  cyclic  macro-operations  from  planning.  If  providing  a  planning  system 
with  such  experience  that  a  more  general  strategy  can  be  inferred,  a  traditional 
planner  might  be  called  eventually  with  all  possible  states  of  a  given  problem  as 
initial  state.  The  effort  of  planning  immediately  for  n  states  with  the  method 
used  by  DPLan  is  sure  more  efficient  than  calling  a  standard  planning  system 
(with  backtracking)  n  times.  An  argument  agains  our  strategy  is,  that  we  can¬ 
not  model  incremental  learning  as  in  PRODIGY  [VCP+95].  (2)  DPlan  is  not 
mtended  primarily  as  efficient  planning  system  but  as  a  method  for  generat¬ 
ing  initial  programs.  We  argue,  that  we  can  bridge  the  gap  between  universal 
planning  and  program  synthesis  by  planning  only  for  problems  with  small  com¬ 
plexity  and  than  generalize  over  the  structure  of  these  problems  (for  example 
from  a  Tower  of  Hanoi  problem  with  3  discs  to  problems  with  an  arbitrary  num¬ 
ber  of  discs) .  (3)  Systems  which  use  domain  axioms  to  guarantee  the  generation 
of  valid  plans  -  as  model-based  planning  systems  [CRT98]  -  make  the  task  of 
domain  and  problem  specification  much  harder  for  the  user  than  DPlan.  For  the 
standard  user  it  is  clearly  more  easy  to  enumerate  all  possible  states  of  a  small 
example  domain  than  to  formulate  a  complete  and  consistent  set  of  axioms. 

To  take  the  load  of  enumerating  all  states  of  a  problem  from  the  user,  we 
could  use  one  of  the  following  two  strategies:  (a)  generate  the  states  automat¬ 
ically  from  an  initial  state  using  the  following  algorithmical  idea:  generate  the 
solution  path  from  initial  state  to  goal  by  forward-application  of  rules.  For  all 
states  lying  on  the  solution  path  which  are  not  yet  regarded:  take  these  states 
as  new  initial  states  and  calculate  their  solution  path.  Terminate  if  no  solution 
path  contains  states  which  were  not  yet  regarded^®;  or  (b)  only  provide  DPlan 
with  an  goal  state  and  with  the  set  of  objects  (for  example  blocks  A,  B,  C) 
to  be  considered,  calculate  predecessors  as  specified  in  table  1  but  with'the  fol- 

® Remark:  see  theorem  1  in  [BF97];  for  operators  with  a  fixed  upper  number  of  parameters 
K,  time  effort  is  polynomia!  in  A;. 

Remark:  This  is  a  first  idea  and  not  really  thought  through. 
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lowing  modifications:  generate  all  operation  candidates  by  instantiate  operators 
with  all  predefined  objects  and  check  admissibility  of  a  new  state  S'  by  forward 
application  Res{o^  S')  =  S  for  the  current  state  S. 

3.5  Comparison  with  Other  Planners 

To  discuss  DPlan  in  relation  to  other  planning  systems,  we  first  introduce  some 
classification  criteria  [RN95,  AHEQO,  Wel99b]: 

♦  state  space  vs.  plan  space.  Planning  can  either  be  described  as  search 
in  state  space  or  as  search  in  the  space  of  partial  plans.  State  based 
planners  introduce  new  (intermediate)  states  by  applying  (instantiated) 
operators  to  a  current  state.  Planning  terminates  if  an  operator  sequence 
transforming  the  initial  state  into  the  goal  state  is  found.  That  is,  a  plan 
is  a  path  though  the  search  tree  with  states  as  nodes  and  operators  as 
arcs.  Plan  space  planners  search  though  the  space  of  (partial)  plans.  An 
initial  plan  (“null  plan”)  consists  of  the  initial  state  -  as  preconditions 
which  are  initially  true  -  and  the  goal  statement  -  as  postconditions  (ef¬ 
fects)  which  finally  must  be  true.  A  partial  plan  is  expanded  by  refinement 
operators  and  modification  operators:  Refinement  operators  add  ordering 
constraints  (putting  one  plan  step  before  another)  and  binding  constraints 
for  variables,  modification  operators  introduce  new  planning  steps.  Plan¬ 
ning  terminates  if  the  plan  is  complete  (every  preconditions  of  every  step 
is  achieved  by  some  other  step)  and  consistent  (there  are  no  contradictions 
in  ordering  and  binding  constraints). 

♦  progression  vs.  regression.  Progression  planners  search  forward  from 
the  initial  state  to  the  goal;  regression  planners  search  backwards  -  back¬ 
ward  planning  does  not  rely  on  a  complete  description  of  states.  The  no¬ 
tion  of  forward /backward  search  is  not  strictly  applicable  to  plan-spaced 
planners. 

♦  linear  vs.  non-linear.  Linear  plans  do  not  allow  for  interleaving  of  goals. 
That  is,  if  a  subgoal  is  selected,  first  all  the  steps  to  solve  this  subgoal  are 
calculated  after  the  next  subgoal  is  considered.  Linear  planners  are  not 
complete  (they  can  for  example  not,  or  not  efficiently,  solve  the  Sussman 
anomaly  [Sus75])! 

♦  total  vs.  partial  order.  Total  order  planners  return  a  list  (sequence)  of 
planning  steps;  partial  order  planners  allow  to  leave  (independent)  plan¬ 
ning  steps  unordered. 

♦  domain  and  problem  specification.  A  problem  is  usually  presented 
by  an  initial  state  and  a  (set /conjunction)  of  problem  solving  goals.  The 
specification  of  operators  is  sometimes  called  domain  theory.  In  STRIPS 
[FN71b]  (initial)  states  are  represented  as  conjunctions  of  function-free 
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ground  literals,  goals  are  represented  as  conjunctions  of  function-free  liter¬ 
als  (which  can  contain  implicitly  existentially  quantified  variables).  State 
descriptions  must  not  be  complete.  Operators  are  represented  by  precon¬ 
ditions,  an  action  description  (name),  and  effects.  Preconditions  are  a 
conjunction  of  atoms,  effects  conjunctions  of  literals.  Effects  are  repre¬ 
sented  as  ADD-  and  DEL-  lists  (some  times  as  a  single  list  of  literals  with 
the  DEL-effects  negated).  Instantiated  operators  are  called  actions.  Op¬ 
erators  containing  variables  are  called  operator  schemes.  If  all  variables  of 
an  operator  can  be  instantiated  in  a  situation  s  in  such  a  way  that  all  pre¬ 
conditions  are  true,  the  operator  is  called  applicable.  Situation  calculus 
[McC98]  allows  to  represented  problems  and  operators  using  the  full  ex¬ 
pressiveness  of  first-order  logic.  All  predicates  (as  part  of  state,  goal  and 
action  descriptions)  have  a  situation  variable  as  (additional)  argument. 
Currently,  most  planning  systems  use  (at  least  a  subset  of)  ADL  (action 
description  language)  [Ped89]  to  represent  operators.  Using  ADL  gives 
us  most  of  the  expressiveness  of  situation  calculus  without  sacrificing  the 
efficiency  of  using  STRIPS  operators.  ADL  allows  for  conditional  effects 
(eliminating  premature  commitment  to  an  operator  as  source  of  ineffi¬ 
ciency),  negated  and  disjunctive  goals,  universal  quantification.  Addition¬ 
ally,  updating  of  variable  values  and  application  of  functions  is  allowed. 

We  can  now  classify  some  well-known  planning  systems: 

♦  Classic  work  in  planning  is: 

—  GPS  [NS61]:  a  state-space,  linear,  total-order  planner;  it  uses  means- 
end  analysis,  starting  with  the  planning-goals,  attacking  one  goal 
after  the  other  in  arbitrary  order,  using  depth-first  search  with  back¬ 
tracking. 

—  QA3  [Gre69]:  models  planning  as  theorem  proving,  using  situation 
calculus.  Another  prominent  approach  to  planning  as  deductive  proof 
problem  was  introduced  by  [MW87]. 

—  STRIPS  [FN71b]:  a  state-space,  linear,  total  order  progression  plan¬ 
ner;  a  similar  approach  is  realized  in  HACKER  [Sus75]. 

—  NOAH  [Sac75]:  was  the  first  partial-order  planner,  working  in  plan- 
space  and  being  non-linear  (i.e.  allows  interleaving  of  goals) 

-  [Wal75]:  introduced  goal-regression  planning;  their  approach  is  based 
on  state-space  and  generates  non-linear  and  total  order  plans;  a  sys¬ 
tem  in  this  tradition  is  for  example  INTERPLAN  [Tat75]. 

•  The  next  generation  of  planners: 

NOAH  was  originally  characterized  as  hierarchical  planner.  Today,  hierarchical  planning 
describes  -  knowledge  intensive  -  approaches  which  are  based  on  operators  on  different  levels 
of  abstraction,  as  for  example  ABSTRIPS  [Sac74]  and  hierarchical  task  network  planners 
[ea94] 
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-  In  the  eighties  and  early  nineties  the  main  focus  was  on  plan-space 
partial- order  planning.  Prominent  examples  are:  TWEAK  [ChaST], 
SNLP  [MR91]  and  finally  UCPOP  [PW92],  which  is  an  extension  of 
SNIP  to  ADL;  see  [Wel94]  for  an  overview.  In  contrast  to  state-space 
planners,  these  planners  employ  a  least  committment  strategy:  that 
is,  instantiations  of  variables  and  orderings  of  steps  are  delayed  as 
long  as  possible.  The  central  -  and  computationally  most  complex 
-  aspect  of  these  planners  is  to  detect  threads  and  calculate  causal 
links  between  planning  steps. 

—  PRODIGY  [Vel94]  is  a  state-space,  regression,  non-linear,  total  order 
planner  with  a  powerful  domain  specification  language  (cf.  allowing 
type-hierarchies  and  infinite  datatypes  for  variables).  In  contrast  to 
the  systems  named  above,  the  work  in  PRODIGY  does  not  focus  on 
efficiency  but  on  combining  planning  and  learning  (allowing  variable 
control  strategies  and  learning  of  such  strategies) . 

-  Additionally  to  these  -  so-called  classic  -  planning  systems,  approaches 
for  dealing  with  uncertain  and/or  incomplete  information  were  de¬ 
veloped  (see  [RN95],  Chap.  13  for  an  overview).  Replanning  systems 
monitor  plan  execution,  detect  violations  of  current  preconditions 
and  generate  a  new  plan  starting  from  the  current  state.  Conditional 
planners  (cf.  CNLP  [PS92],  B-PRODIGY  [BV97])  introduce  different 
subplans  for  different  contexts  (i.e.  generate  trees  of  plans).  Reactive 
planners^  as  for  example  universal  planning  [Sch87],  calculate  the  de¬ 
sired  action  for  each  possible  state  of  the  world  (which  is  of  course 
only  possible  in  -  small  -  finite  domains).  An  alternative  to  planning 
in  uncertain  environments  is  the  use  of  reinforcement  learning. 

♦  Current  trends:  A  variety  of  new  planning  approaches  attacking  the  prob¬ 
lem  of  planning  efficiency  are  developed.  There  is  a  renaissance  of  state- 
based  approaches. 

—  GRAPHPLAN  [BF97,  KNH97]:  started  the  series  of  approaches 
which  transfer  techniques  from  other  domains  of  computer  science 
to  planning.  The  concept  of  minimal  flow  in  networks  was  applied  to 
planning  problems:  In  a  first  step,  a  planning  graph  is  constructed 
by  forward  search,  in  a  second  step  a  plan  is  extracted  by  backward 
search.  The  planning  graph  is  organized  in  levels  corresponding  to 
time  steps.  Nodes  are  facts  (single  ground  literals  in  contrast  to 
states)  and  actions.  The  first  level  represents  all  facts  which  are 
true  in  the  initial  state;  the  next  level  represents  all  actions  which 
have  their  preconditions  fulfilled  by  the  initial  facts  and  %o-op”  ac¬ 
tions  (doing  nothing  preserves  facts);  the  next  level  contains  all  add- 
and  del-effects  of  the  actions  and  the  preserved  facts.  Edges  are  in¬ 
troduced  only  between  immediate  succeeding  levels:  preconditions 
edges  from  facts  to  actions  and  add-  and  del-edges  from  actions  to 
facts.  Additionally,  a  (incomplete)  set  of  exclusion  relations  among 
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planning  graph  nodes  is  calculated  (mutex)  for  interfering  or  compet¬ 
ing  actions  and  propositions.  If  all  goal  propositions  are  reached  by 
add-edges  and  are  not  mutex  at  one  level,  plan  expansion  terminates 
with  success  and  we  know,  that  a  valid  plan  exists  for  the  given  prob¬ 
lem.  In  the  next  step,  the  planning  graph  is  searched  backward  for 
a  valid  plan  (a  “flow”  from  the  last  level  to  the  initial  propositions). 
That  is,  a  valid  plan  is  a  subgraph  of  the  planning  graph.  Indepen¬ 
dent  actions  are  not  ordered  but  returned  as  a  set  if  they  occur  at 
one  level,  that  is,  a  partial  order  plan  is  returned.  The  original  algo¬ 
rithm  was  based  on  STRIPS-like  operators  [BF97];  the  planner  IPP 
[KNH97]  is  an  extension  to  an  ADL  subset.  To  use  our  terminology 
from  above:  graphplan  approaches  are  state-based,  nonlinear,  partial 
order  regression  planners. 

—  SAT-planners,  as  BLACKBOX  [KS98]:  argue  that  planning  is  a  spe¬ 
cial  kind  of  theorem-proving  and  that  first-order  theorem-proving 
does  not  scale  well.  They  propose  to  model  planning  in  the  frame¬ 
work  of  propositional  satisfiability  testing  rather  than  to  use  special¬ 
ized  planning  algorithms.  Similar  to  GRAPHPLAN,  SAT-planning 
relies  on  a  propositional  planning  graph.  States  and  actions  are  rep¬ 
resented  as  (propositional)  logical  clauses.  A  plan  corresponds  to  a 
model  (i.e.  truth-assignment)  that  satisfies  a  set  of  logical  constraints 
representing  initial  state,  goal  state  and  domain  axioms.  Domain  ax¬ 
ioms  characterize  the  consistency  of  states  and  legal  state  transitions 
(i.e.  operator  applications).  Currently,  diflferent  representations  for 
planning  graphs  are  investigated,  for  example  graphplan-based  en¬ 
codings  and  state-based  encodings.  Representing  states  rather  than 
single  literals  as  nodes  seems  most  promising.  An  efficient  (but  in¬ 
complete)  method  for  finding  plans  is  to  use  local  stochastic  search 
(as  Walksat). 

-  Model-based  planners:  are  a  current  approach,  especially  for  plan¬ 
ning  in  non-deterministic  domains.  Model-based  planners  generate 
universal  plans  in  an  reasonably  efficient  way  by  encoding  plans 
as  OBDD’s  (ordered  binary  decision  diagrams)  and  using  symbolic 
model  checking  to  explore  large  state  spaces  [CRT98].  Application 
of  symbolic  model  checking  to  deterministic  planning  problems  is 
reported  in  [DGR98]. 

DPlan  is  a  state-based  non-linear  total  order  regression  planner.  Similar  to 
universal,  conditional  and  model-based  planners  it  does  not  rely  on  the  pre¬ 
sentation  of  an  initial  state  but  works  on  sets  o/ states  instead.  In  contrast  to 
model-based  planners  but  similar  to  conditional  planners,  we  specify  operators 
in  STRIPS  or  a  subset  of  ADL  (and  not  predefine  all  possible  actions)  and  we 
do  not  rely  on  domain  axioms.  Currently,  domain  specifications  for  DPlan  allow 
for  conditional  effects,  but  we  want  to  extend  it  to  a  larger  subset  of  ADL. 

Our  planner  is  sound  and  complete  and  termination  (with  a  valid  plan  or  a 
partial  plan)  is  guaranteed  (which  is  not  in  general  true  for  plan-based  partial- 
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order  planners). 

Planning  effort  is  polynomial  in  the  number  of  states.  This  statement  is 
somewhat  unfair  when  comparing  DPlan  with  classical  planners,  because  we 
do  not  consider  the  extraction  of  a  single  plan  (for  a  given  initial  state)  in 
our  analysis  (see  section  3.3,  plan  execution).  In  a  loose  way,  the  (backward) 
generation  of  a  plan  in  DPlan  corresponds  to  the  (forward)  construction  of  a 
planning  graph,  which  can  also  be  done  in  polynomial  time. 

Finally,  DPlan  can  guarantee  to  calculate  optimal  plans.  This  is  true,  be¬ 
cause  our  planning  algorithm  is  basically  a  breadth-first  search. 

Other  than  all  planners  mentioned  above,  the  main  motivation  for  DPlan 
was  not  to  develop  a  sound,  complete  and  eflScient  planner  but  to  provide  initial 
programs  for  program  synthesis  (see  section  2).  It  is  recognized  in  the  plan¬ 
ning  community  that  there  is  a  close  relation  between  conditional  planning  and 
program  synthesis  (see  [RN95],  Chap.  13,  p.  411).  Our  main  interest  is  to 
cross-fertilize  both  fields:  using  planing  methods  to  provide  initial  programs  for 
program  synthesis  and  using  program  synthesis  to  provide  a  technique  for  learn¬ 
ing  cyclic  macro-operators  -  thereby  making  universal  planning  more  efficient. 


3.6  Empirical  evaluation 

Although  it  is  not  really  meaningful  to  compare  running  tiines  of  planning  al¬ 
gorithms  which  are  implemented  in  different  programming  languages  (LISP  for 
PRODIGY,  UCPOP  and  DPLAN;  C  for  GRAPHPLAN  and  IPP)  and  using  dif¬ 
ferent  amounts  of  information  (single  initial  states  vs.  sets  of  states  for  DPlan 
and  universal  planners;  domain  axioms  for  model-based  approaches'^)  we  are 
planning  to  run  experiments  using  the  domains  of  the  AIPS-98  planning  com¬ 
petition  (see  http: //www. cs . cmu.edu/ *  aips98/,  and  the  artificial  domains 
from  [BW94]  -  see  also  empirical  results  in  [BF97,  KNH97])^^.  While  the  com¬ 
parison  of  the  absolute  values  for  performance  does  not  give  any  valid  infor¬ 
mation,  a  comparison  of  the  shape  of  the  problem  size/time  plots  can  be  of 
interest.  In  particular,  it  should  support  the  theoretical  analysis  of  planning  ef¬ 
ficiency  given  above  and  give  empirical  insight  in  the  scope  of  problems  solvable 
by  DPlan.  Furthermore,  we  will  use  these  planning  domains  to  identify  and 
illustrate  meaningful  possibilities  for  cyclic  macro  generation. 


We  can  ignore  the  use  of  domain-specific  control  knowledge  cf.  for  PRODIGY. 

Remark:  Hopefully,  the  comparison  will  be  supported  by  participants  of  our  student 
project  in  summer  term  99. 
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Chapter  4 

Transformation  of  Plans  to 
Programs  —  Bridging  the 
gap  between  planning  and 
program  synthesis 

As  described  in  chapter  2,  our  program  synthesis  system  constructs  cyclic  macros 
by  generalizing  over  unifiable  substructures  of  a  initial  program.  To  provide  an 
input  to  the  synthesis  system,  the  original  output  of  DPlan  has  to  be  trans¬ 
formed  into  the  format  of  such  initial  programs.  Transformation  of  the  original 
DPLan  output  -  the  minimal  spanning  tree  -  currently  is  fully  implemented  for 
linear  plans  and  partially  for  plan-trees  (for  example  the  tower  problem  pre¬ 
sented  at  the  end  of  section  3.2,  see  figure  3.5).  As  described  in  chapter  3, 
the  current  implementation  of  DPlan  cannot  deal  with  universal  quantification, 
infinite  types  and  application  of  functions  in  operators.  Therefore,  planning  for 
list  and  number  problems  is  currently  only  possible  in  a  limited  way. 

Because  transformation  of  plans  into  initial  programs  is  still  work  in  progress, 
we  will  present  our  approach  only  informally  and  give  examples  together  with 
an  identification  of  open  problems  in  plan  transformation  instead.  An  informal 
algorithm  for  transforming  D-plans  into  initial  programs  is  given  in  table  4.1, 
The  motivation  for  each  transformation  step  and  an  illustration  for  the  dear- 
block  problem  was  given  in  chapter  2.  Note  that  our  idea  to  linearize  plans  - 
unification  of  branches  with  identical  operations  and  calculating  intersections 
for  conjunctions  of  predicate  -  corresponds  to  concepts  introduced  in  inductive 
logic  programming  [LD94]  and  explanation  based  learning  [MKKC86,  BV96]. 
Furthermore,  dealing  with  a  sequence  of  different  states  in  one  path  corresponds 
roughly  to  the  concept  of  subsumption  in  ILP. 
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Table  4.1:  Informal  algorithm  for  transforming  D-plans  into  initial  programs 
(Steps  marked  with  (*)  are  under  investigation,  that  is,  currently  it  is  not  proven 
that  these  steps  preserve  the  semantic  of  a  D-plan.) 

1.  If  the  plan  is  linear  then 

(a)  Rewrite  Constants.  Replace  constants  by  constructive  expres¬ 
sions:  recursively  apply  the  rewrite-rules  given  in  the  domain  speci¬ 
fication  to  D-plan. 

(b)  Determine  Relevant  Predicates.  Only  keep  such  predicates  of  a 
state  which  occur  in  the  ADD-list  of  the  backward  applied  operation 
for  each  state-node  in  D-plan. 

(c)  Generate  binary  tree.  For  each  predicate-node  introduce  a  left 
successor  s  covering  the  case  that  the  predicates  are  true.  If  no 
further  operation  is  given,  terminate  the  path  with  an  asterix  as  last 
right  successor  of  a  predicate-node. 

(d)  Rewrite  into  program  term.  Transform  the  plan  into  the  syntax 
of  program  terms  (introduce  conditional  expressions), 

2.  else 

(a)  For  each  linear  subplan:  Rewrite  Constants. 

(b)  Determine  Relevant  Predicates. 

(c)  Unify,  branches  with  identical  operations  and  replace  the  preceding 
predicate-nodes  by  their  intersection.  (*) 

(d)  If  the  plan  is  now  linear 
then  Generate  binary  tree. 

else  For  each  linear  subplan:  Generate  binary  tree.  (*) 

(e)  Rewrite  into  program  term. 
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Table  4.2:  Linear  recursive  macro-operations  (here  represented  in  LISP  syntax) 

;  clear  a  block  x:  puttable  all  block  topof  x 
(defun  clearblock  (x  s)  =  (cond  ((cleartop  x)  s) 

(T  (puttable  (topof  x)  (clearblock  (topof  x)  s))))) 
;  build  a  tower:  put  next  block  on  current  topof  tower  —  i.e.  specified  base 
;  (baseof  x)  has  to  be  guaranteed  to  be  cleartop! 

(defun  putblocks  (x  tw)  =  (cond  ((on  x  (baseof  s))  tw) 

(T  (put  X  (putblocks  (baseof  x)  tw))))) 

;  unload  objects  o  from  vehicle  v;  (empty  o)  indicates  that  there  are 
;  no  more  objects  at  the  current  position 
(defun  unloados  (o  d)  =  (cond  ((at  o  d)  d) 

(T  (unload  o  (unloados  (next  o)  d))))) 

;  load  objects  o  into  vehicle  v;  assumes  infinite  capacity  for  v 
(defun  loados  (o  v)  =  (cond  ((insideR  o)  v) 

(T  (load  0  (loados  (next  o)  v))))) 


4.1  Inferring  linear-recursive  macro-operators 

If  the  minimal  spanning  tree  of  a  problem  domain  results  in  an  linear  order  of 
states  and  involves  only  one  operator,  the  plan  can  easily  generalized  to  a  cyclic 
macro-operation  corresponding  to  a  linear  recursion.  A  typical  example  for  a 
linear  macro  is  the  clearblock  function  discussed  in  chapter  2. 

In  general,  linear  recursive  macros  can  be  generalized  from  and  applied  to 
all  problem  domains  which  involve  the  repeated  application  of  an  operation  to 
a  data  structure  which  is  reduced  by  each  application  step.  For  the  clearblock 
problem  the  operation  puttable  has  to  applied  for  each  block  lying  on  top  of  the 
block  which  has  to  be  cleared.  Other  examples  for  linear  domains  are:  building  a 
tower  by  putting  one  block  upon  another  block  if  the  blocks  currently  considered 
are  guaranteed  to  be  clear,  and  loading  or  unloading  a  series  of  objects  in  or  from 
a  transportation  vehicle  (rocket,  plane,  truck,  train,  or  briefcase).  A  sample  of 
linear  recursive  macros  is  given  in  table  4.2. 

4.1.1  Unstacking  and  building  a  tower 

The  domain  specification  for  building  a  tower  is  given  in  table  4.3.  The  original 
output  of  DPlan  is  given  in  figure  4.1.  Note,  that  we  defined  put  with  an 
conditional  effect  as  above.  But  for  the  given  problem  domain  the  case  that  x 
is  lying  on  another  block  (and  not  on  the  table)  never  occurs. 

Plan  transformation  into  an  initial  program  starts  with  replacing  constants 
by  constructive  expressions.  We  apply  a  rewrite-rule  converse  to  clearblock 
-  the  block  which  must  be  put  before  another  block  in  accordance  with  the 
goal  statement  is  the  baseof  this  block  (see  figure  4.2).  The  put  operator  adds 
on{x,  y)  as  primary  effect  -  for  this  simple  linear  domain  the  side  effect  ct{z)  is 
never  produced.  Thus,  the  relevant  predicate  at  each  state  node  is  on  (a?,  y)  only. 
Figure  4.3  presents  the  binary  planning  tree  for  putblocks.  In  this  transformation 
step,  the  leaf  node  is  not  reduced  (see  arguments  and  heuristics  in  section  2.4). 

The  binary  tree  can  now  easily  be  transformed  into  a  program  term  (initial 
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Table  4.3;  Domain  specification  for  stacking  blocks  {putblocks) 

;  Linear  put  (analogous  to  clearblock) 

;  putting  one  block  upon  another  when  each  block  is  clear  when  considered 
(Make-Package  ^pd-put) 

(Export  ^(states  goal  rules  is-c-pred  c-eq  )) 

(setq  states  ^( 

((on  a  b)  (on  b  c)  (ct  a)) 

((on  b  c)  (ct  a)  (ct  b)) 

((ct  a)  (ct  b)  (ct  c)) 

)) 

(setq  goal  »(  (on  a  b)  (on  be))) 

(setq  rules 
’ (  (rule  put 

(if  (  ct  (>  x)  ) 

(  ct  (>  y)  )  ) 

(then  (put  (<  x)  (<  y)  )  ) 

(conq  (cond  ((on  «  x)  (>  z))  (add  ((on  «  x)  (<  y))  (ct  «  z))) 

del  ((ct  (<  y))  (on  (<  x)  (<  z))))) 
(  T  (  add  ((on  (<  x)  (<  y))) 
del  ((ct  «  y)))  )  ) 

)  ) 

) 

)) 


;  Predicates  which  indicate  constructive  rewriting 
(defun  is-c-pred  (p-name) 

(equal  p-name  ^on) 

) 

;  (on  X  y)  ==  [y  =  (next  x)] 

(defun  c-eq  (p) 

(list  (nth  2  p)  (list  ^baseof  (nth  Ip)))  ;  assoc-list 

) 
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Figure  4,1:  Output  of  DPlan  for  putblocks 
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Figure  4.2:  Rewritten  constants  for  putblocks 
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Figure  4.3:  Binary  tree  for  putblocks 


program)  by  interpreting  predicate  nodes  as  conditions,  left  branches  as  then- 
and  right  branches  as  else-case: 

G  =  g{on{xjbaseof{x)),SyPut{x^ 

g{on{baseof{x),baseof{baseof{x))),s^put{baseof{x)^ 

m) 


or  in  LISP  syntax 

G  =  {cond{{on  x  {baseof))  s) 

(T(put  X 

{cond{{on{baseof  x)[baseof  {baseof  a?)))  s) 
{T{put{baseof  x) 
omega)))))). 

The  initial  program  can  be  folded  in  the  recursive  macro  given  in  table 
4.2.  Note,  that  we  named  s  into  tw  (for  ^'current  tower”)  for  more  intuitive 
readability.  Remember,  that  the  situation  variable  s  is  introduced  to  represent 
the  current  situation  (state)  which  is  returned  if  the  predicates  at  the  parent 
node  are  true  in  this  state. 

4. 1,2  Unloading  and  loading  objects 

Veloso  [VC93a]  proposed  the  rocket  and  a  more  general  transportation  do¬ 
main.  Transportation  problems  typically  involve  the  loading  of  objects  into  a 
transport- vehicle  and  their  unloading  at  a  given  destination.  Plan  construction 
for  transportation  problems  often  relies  on  interleaving  of  subgoals  to  generate 
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optimal  plans  -  first  loading  all  objects  at  the  current  position  and  then  driving 
to  the  destination  instead  of  loading  one  objects,  driving  to  the  destination, 
driving  back,  transferring  the  next  object  and  so  on.  If  resources  (fuel)  have  to 
be  taken  into  regard  or  if  a  transport  vehicle  can  only  drive  in  one  direction  as  in 
the  rocket  domain,  handling  all  objects  at  a  location  even  becomes  necessary  for 
construction  of  valid  plans.  Applying  load  and  unload  macros  in  these  domains 
could  reduce  planning  effort  dramatically. 

In  the  next  section  we  will  see,  that  -  although  rocket  is  a  relatively  simple 
structured  domain  -  it  is  not  strictly  linear  when  planning  without  restricting 
the  order  of  handling  objects.  The  rocket  domain  involves  the  execution  of  two 
consecuting  linear  macros  load  and  unload  and  of  an  move  operator  inbetween. 
In  the  following  we  will  deal  with  reduced  problems:  planning  to  load  or  unload 
objects  from  a  vehicle.  The  problem  specifications  are  given  in  tables  4.4  and 
4.5.  Because  we  will  discuss  macro  generation  for  the  rocket  domain  in  the  next 
section,  the  predicate  names  and  rule  definitions  are  taken  from  this  domain. 
Note,  that  we  do  not  consider  the  state  ((insideR  o2)  (at  ol  B)  (atR  B))  for 
unload  and  the  state  ((insideR  o2)  (at  ol  A)  (atR  A))  for  load.  That  is,  we 
already  imply  an  ordering  of  objects  in  the  state  definition.  Otherwise,  the 
resulting  plan  would  not  be  linear.  Furthermore,  the  rewrite  rules  for  load  and 
unload  are  converse  (as  for  clearblock  and  putblocks)  to  guarantee  that  the  basic 
case  (the  “bottom’^  object,  which  is  not  rewritten  constructively)  will  appear  in 
the  root  of  the  linear  plan. 

The  DPlan  outputs  for  unload  and  load  are  given  in  figures  4.4  and  4.5.  The 
binary  trees  are  given  in  figures  4.6  and  4.7.  The  resulting  program  terms  are: 

G  =  g  {at  {ol  B)  ^  B,  unload  {ol, 

g{at{nexi{ol)B),  B,  unload{next{ol), 

m) 


for  unload  and 

G  =  g{insideR{o2),V,load{o2, 

g{insideR{next{o2)),  V,  load{next{o2) , 
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Table  4.4:  Domain  specification  for  unloading  objects 

unload  (basic  macro  for  transportation  domains) 


unload  all  objects  from  a  vehicle 


(Hake-Package  >pd-unload) 

(Export  > (states  goal  rules  is-c-pred  c-eq)) 

(setq  states  ’( 

((at  ol  B)  (at  o2  B)  (atR  B)) 

((insideR  ol)  (at  o2  B)  (atR  B)) 

((insideR  ol)  (insideR  o2)  (atR  B)) 

)) 

(setq  goal  >((at  ol  B)  (at  o2  B))) 

(setq  rules 

*(  (rule  unload 

(if  (insideR  (>  x))  (atR  (>  y))) 
(then  (unload  (<  x))) 

(conq  (add  ((at  (<  x)  (<  y))) 
del  ((insideR  (<  x))) 

)  ) 

) 

)) 


;  Predicates  which  indicate  constructive  rewriting 
(defun  is-c-pred  (p-name) 

(or  (equal  p-name  ^at)  (equal  p-name  ^insideR)) 

) 

;  (at  ol  x)  and  (at  o2  x)  ==>  o2  =  next(ol) 

;  (insideR  ol)  and  (insideR  62)  =>  o2  =  next(ol) 

(defun  c-eq  (p) 

(cond  ((equal  (nth  1  p)  >o2) 

(list  (nth  1  p)  ’(next  ol))  ;  assoc-list  (o2  (next  ol)) 

) 
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Table  4.5:  Domain  specification  for  loading  objects 

load  (basic  macro  for  transportation  domains) 


load  all  objects  from  a  vehicle  (assuming  a  vehicle  with  infinite 
capacity 


(Hake “Package  ’pd-load) 

(Export  ^(states  goal  rules  is“C“pred  c-eq)) 


(setq  states  ’( 

((at  ol  A)  (at  o2  A)  (atR  A)) 

((insideR  ol)  (at  o2  A)  (atR  A)) 
((insideR  ol)  (insideR  o2)  (atR  A)) 

)) 

(setq  goal  > ((insideR  ol)  (insideR  o2))) 

(setq  rules 
’ (  (rule  load 

(if  (at  (>  x)  A)  (atR  A)) 

(then  (load  (<  x))) 

(conq  (add  ((insideR  (<  x))) 
del  ((at  (<  x)  A)) 

)  ) 

) 

)) 


;  Predicates  which  indicate  constructive  rewriting 
(defun  is“C“pred  (p-name) 

(or  (equal  p-name  ^at)  (equal  p-name  ^insideR)) 

) 

;  (at  ol  x)  and  (at  o2  x)  =>  ol  =  next(o2) 

;  (insideR  ol)  and  (insideR  o2)  =>  ol  =  next(o2) 

(defun  c-eq  (p) 

(cond  ((equal  (nth  Ip)  ^ol) 

(list  (nth  1  p)  ^ (next  o2))  ;  assoc-list  (ol  (next  o2)) 

) 

)) 
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[quit  I 
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Figure  4.4:  Output  of  DPlan  for  unload 
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Figure  4.5;  Output  of  DPlan  for  load 
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Figure  4.7:  Binary  tree  for  load 
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4.2  Complex  cyclic  macros:  non-linear  struc¬ 
tures 

Examples  for  non-linear  structures  are  tower  (presented  in  chapter  3.2  and  dis¬ 
cussed  in  [Wys87]),  rocket  di,nd  other  transportation  domains  [VC93a],  and  Hanoi. 
A  non-linear  D-plan  not  necessarily  implies  that  the  resulting  cyclic  macro  has 
to  be  non-linear.  The  reason  for  this  can  be  found  in  function  theory  [FH88]: 
A  given  syntactical  realization  of  a  function  does  not  necessarily  correspond 
with  its  semantic  complexity  [Hin78,  Odi89].  For  example,  we  can  rewrite  a 
tree-recursive  function  for  calculating  Fibonacci  numbers  into  a  linear  recursive 
structure  (by  introducing  an  additional  variable),  or  there  exist  loop-programs 
(corresponding  to  linear  recursions)  for  Hanoi  problems  [Er83,  Pet84].  Similarly, 
a  given  sample  of  problem  solutions  -  as  represented  in  a  D-plan  -  may  have 
characteristics  which  make  it  possible  to  rewrite  its  given  syntactical  form  into 
a  computationally  less  complex  structure. 

We  can  think  of  three  possible  strategies  to  deal  with  non-linear  DPlan 
outputs  which  we  want  to  incorporate  in  our  system: 

♦  Using  already  acquired  linear  macros:  We  can  provide  our  system  first  with 
simple  linear  problems  and  use  the  generalized  linear  macros  when  plan¬ 
ning  for  more  complex  problems.  For  example,  using  the  unload  and  load 
macros  for  solving  the  rocket  problem  results  in  a  linear  plan:  load  objects, 
move  rocket,  unload  objects.  This  kind  of  “hierarchical”  macro  learning 
has  two  problems:  (a)  we  need  a  teacher  which  provides  the  system  with 
problems  in  such  a  sequence  that  hierarchical  learning  is  possible;  (b)  dur¬ 
ing  planning  not  only  the  operators  given  in  the  domain  specification  but 
additionally  the  macro  memory  has  to  be  searched  and  applicability  of 
macros  has  to  be  checked. 

Of  course,  the  second  problem  has  to  be  addressed  anyway  when  we  want 
to  apply  cyclic  macros  to  make  planning  more  efficient.  Checking  if  a 
macro  is  applicable  can  be  performed  in  two  ways:  either  we  index  macros 
with  semantical  information  (the  predicates  they  fulfill  and  the  domain  in 
which  they  were  learned)  or  we  use  pattern-matching  between  partial  D- 
plan  trees  and  unfolded  cyclic  macros  (i.e.  an  initial  program  generated 
by  expanding  macro  to  a  given  depth) 

♦  Rewriting  non-linear  structures  into  linear  structures  if  possible:  That 
is  the  strategy  we  currently  investigate.  After  constructively  rewriting 
each  linear  path  of  the  plan,  we  try  to  unify  different  planning  paths. 
If  the  resulting  structure  is  still  a  tree,  we  rewrite  a  “side-ordering”  of 
paths  (corresponding  to  a  “case”  statement)  into  a  nested  “sub-ordering” 
(corresponding  to  nested  if-then-else  statements)  -  that  is,  we  convert  a 
partial  order  into  an  arbitrary  total  order. 

♦  Generating  initial  programs  from  non-linear  plans:  While  linear  macros 


^Remark:  Stefan  Bohm  is  investigating  this  strategy  in  his  diploma  thesis. 
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are  computationally  more  efficient  than  tree-recursive  ones,  they  are  not 
necessarily  more  efficient  on  the  level  of  representation.  Furthermore,  a 
more  compact  (tree-)  recursive  macro  is  often  easier  to  understand  by 
humans  than  an  optimized  version  because  it  is  formulated  in  a  “more 
natural”  way  (compare  for  example  the  tree  and  the  linear  recursive  ver¬ 
sions  of  fibonacci  or  Hanoi) . 

•  For  the  last  two  cases  we  may  either  generalize  a  “complex”  macro,  or  try 
to  learn  encapsulated  linear  macros  (as  load  and  unload)  in  the  context  of 
a  more  complex  problem  (as  rocket).  The  second  strategy  has  to  be  based 
on  a  principle  for  segmenting  a  problem  in  independent  subproblems  (a 
syntactical  approach  for  such  segmentations  is  realized  for  example  in 
ALPINE  [Kno90]). 

The  rest  of  this  section  will  be  very  superficial.  Because  transformation  for 
non-linear  plans  is  not  fully  implemented  yet^  we  just  present  the  original  DPlan 
outputs  and  discuss  some  open  problems. 

4.2.1  Building  a  tower  of  alphabetically  ordered  blocks 

Koza  [Koz92]  synthetisized  a  program  for  building  a  tower  of  blocks  in  a  pre¬ 
defined  order  using  genetic  programming  -  that  is,  he  provided  the  learning 
algorithm  with  a  training  set  of  initial  states  and  the  desired  goal  and  inferred 
the  function  by  search  in  the  space  of  possible  programs  (see  figure  4.8). 

This  program  does  not  guarantee  that  for  each  possible  input  the  shortest  oper¬ 
ation  sequence  is  executed:  for  initial  states  which  correspond  to  towers  sorted 
reverse  to  the  goal  tower  it  is  not  necessary  to  put  all  blocks  on  the  table  first. 
Instead,  the  tower  can  just  reversed  by  exploiting  the  side-effect  of  put  that  a 
block  gets  cleared  if  the  top  of  it  is  put  on  another  block. 

A  program  which  builds  a  tower  from  arbitrary  input  states  with  optimal 
operation  sequences  is  given  in  appendix  B.  The  main  function  shows  one 
possible  realization  of  a  cyclic  macro  for  solving  the  tower  problem: 

(defun  tower  (1) 

(cond  ((is“tower  1)  1) 

((and  (exists-tower  1) 

(exist-free~neighbors  1)) 

(tower  (put  (scndgreatest  1)  (greatest  1)  1))) 

(T  (tower  (puttable  (topof  (greatest-no~tower  1))  1))) 

)) 


To  generalize  a  macro  corresponding  to  this  program,  it  would  be  necessary 
to  “invent”  predicates  (is-tower,  exist s-tower,  exist s-free-neighbors) 
and  introduce  more  than  one  selector  function  (not  only  topof  as  used  in  the 
clearblock  ma,CTO,  but  also  greatest,  scndgreatest,  and  great est-no-tower). 
Predicate  invention  is  an  established  method  in  inductive  logic  programming 


^Remark:  I  hope  to  be  finished  with  a  first  prototype  by  Mai  1999. 
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program  synthesized: 

(EQ  (DU  (MT  CS)  (NOT  CS))  (DU  (MS  NN)  (NOT  NN))) 

a  more  readable  version: 
equal  ( 

do  move-tO’table(x)  until  not  (current-stack), 

do  move-to-stack(next-necessary-block)  until  not  (next-necessary-block) 


Genetic  Programming: 

•  representing  programs  as  function  terms 

•  cross-over  (syntactically  correct)  terms 

•  evaluate  programs  by  their  performance  on  a  set  of  examples 


Figure  4,8:  Synthesizing  tower  with  genetic  programming 


[FYar],  Therefore,  it  would  be  interesting  to  try  to  incorporate  this  idea  in  our 
plan  transformation  approach. 

Another  possible  macro^  is:  apply  putfx,  y)  for  the  desired  goal  order  of 
blocks  beginning  from  the  base  and  apply  cleartop  to  each  block  which  is  argu¬ 
ment  of  put 

;  1  is  the  (reversed)  goal  sorting,  f.e.  (C  B  A) 

;  s  is  initialized  with  an  initial  state  and  modified  by  put  and  clearblock 
;  clearblock  is  a  recursive  function  using  puttable 
(defun  tower  (Is) 

(cond  ((null  1)  s) 

(T  (tower  (cdr  1) 

(put  (clearblock  (cadr  1)  s)  (clearblock  (car  1)  s)  s))) 

)). 


The  strategy  we  currently  investigate  will  generate  non  of  these  macros  but 
a  single  complex  function  (see  discussion  of  the  rocket  domain  below). 

4.2.2  Rocket  problems  for  arbitrary  numbers  of  objects 

The  DPlan  domain  specification  for  rocket  is  given  in  table  4.6. 

There  exists  a  second  legal  state  fulfilling  the  goal  -  ((at  ol  B)  (at  o2  B) 
(atR  A))  -  which  we  currently  do  not  consider.  The  output  of  DPlan  for  the 
rocket  domain  is  given  in  figure  4.9.  If  we  allow  different  sequences  to  load  or 
unload  objects  in  the  definition  of  states,  the  state  ordering  is  not  completely 
linear.  There  is  always  a  branch  dealing  with  an  alternative  ordering  -  but 
this  branch  is  always  pruned  because  the  following  states  are  already  dealt  with 
in  another  branch.  This  observation  leads  us  to  the  following  hypothesis  for  a 

^proposed  by  Fritz  Wysotzki  1998 
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Table  4.6:  The  rocket  domain 

one-way  rocket  transportation  problem^  Veloso  &  Carbonell  93 


two  locations  A  and  B  and  a  oneway-st  between 

transporter:  R  (rocket)  can  load  objects  (two  objects  ol ,  o2) 


(Make-Package  ’pd-rocket) 
(Export  ’ (  states  goal  rules 

is-c-pred  c-eq  )) 


(setq  states  ’( 

((at  ol  B)  (at  o2  B)  (atR  B)) 

((insideR  ol)  (at  o2  B)  (atR  B))  ;  symmetric  states 

((insideR  o2)  (at  ol  B)  (atR  B))  ; 

((insideR  ol)  (insideR  o2)  (atR  B)) 

((at  ol  A)  (atR  A)  (at  o2  B)) 

((insideR  ol)  (atR  A)  (at  o2  B)) 

((at  o2  A)  (atR  A)  (at  ol  B)) 

((insideR  o2)  (atR  A)  (at  ol  B)) 

((at  ol  A)  (at  o2  A)  (atR  A)) 

((insideR  ol)  (at  o2  A)  (atR  A))  ;  symmetric  states 

((insideR  o2)  (at  ol  A)  (atR  A))  ; 

((insideR  ol)  (insideR  o2)  (atR  A))  )) 

(setq  goal  ^((at  ol  B)  (at  o2  B))  ) 

(setq  rules 
’ (  (rule  move 

(if  (atR  A)  ) 

(then  (moveR)) 

(conq  (add  ((atR  B)) 

del  ((atR  A))  )  )  ) 

(rule  unload 

(if  (insideR  (>  x))  (atR  (>  y))) 

(then  (unload  (<  x))) 

(conq  (add  ((at  (<  x)  (<  y))) 

del  ((insideR  (<  x)))  )  )  ) 

(rule  load 

(if  (at  (>  x)  A)  (atR  A)) 

(then  (load  (<  x))) 

(conq  (add  ((insideR  (<  x))) 

del  ((at  (<  x)  A))  )  )  )  )) 


;  Predicates  which  indicate  constructive  rewriting 
(defun  is-c-pred  (p-name) 

(or  (equal  p-name  >at)  (equal  p-name  > insideR)) 

) 

;  (at  ol  x)  cind  (at  o2  x)  =>  o2  =  next(ol) 

;  (insideR  ol)  and  (insideR  o2)  =>  o2  =  next(ol) 

(defun  c-eq  (p) 

(cond  ((equal  (nth  1  p)  ^o2) 

(list  (nth  1  p)  Knext  ol))  ;  assoc-list  (o2  (next  ol)) 

) 
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|«BT  01  B)  (ftT  02  B)  (HTR  B»  | 


I  <IM.imD  0g>  I 


ICdHSIDER  01)  <HT  02  B)  <BTR  B»  | 


[«IHS1DER  02>  (RT  01  B>  <flTR  B)) 


I (UNLOAD  OgT] 


|«IHSIDER  02)  <INSIDER  01)  <nTR  B))  | 
[(HOVER)  I 

[({flTR  fl)  (IMSI^R  02)  (IMSIDER  Dl»  | 


I (LOAD  02)  1 


I (LOAD  01)  I 


((AT  02  fl>  (flTR  H)  (IHSlIgR  01) )|  [((AT  01  H>  (BTR  H)  (IHSIDER  02)T| 


I  [((AT  01  fl)  (AT  re  B)  (BTR  R»  | 


Figure  4.9:  Output  of  DPlan  for  rocket 


linearization  heuristic:  If  an  operation  leads  to  a  leaf  node  and  if  this  operation 
is  also  handled  in  another  branch  a  a  deeper  level,  then  the  operation  and  its 
succeeding  leaf  node  can  be  deleted  from  the  branch. 

If  the  branches  for  alternative  orderings  are  deleted,  all  states  are  in  an  total 
order.  Nevertheless,  generalization  is  not  so  easy  as  for  the  problems  discussed  in 
section  4.1,  because  more  than  one  operator  is  involved.  The  current  implemen¬ 
tation  of  our  synthesis  algorithm  cannot  deal  with  such  inputs:  it  generates  the 
hypothesis  g{at{x ,  B) ,  B ,  unload{x,  rockei{next[x),  B)))  which  cannot  be  main¬ 
tained  when  the  move  operator  appears.  To  solve  this  problem,  our  synthesis 
algorithm  has  to  be  extended  to  subprogram  construction.  We  are  planning  to 
extend  our  synthesis  algorithm  to  subprogram  generation  -  i.e.  allowing  to  infer 
more  than  one  recursive  equation  from  an  initial  program^. 

Two  programs  solving  the  rocket  domain  for  an  arbitrary  number  of  objects 
are  given  in  appendix  C.  Both  programs  cover  all  possible  input  states  -  that 
is  not  only  states  where  all  objects  are  at  location  A  at  the  beginning,  but  also 
problems,  where  some  objects  are  already  in  the  rocket  or  at  the  destination. 
The  first  program  makes  use  of  the  linear  macros  for  unload  and  load,  the  second 
program  is  a  “complex”  macro  for  solving  the  complete  problem.  For  real  world 
applications  it  is  not  acceptable,  that  we  allow  infinite  capacity  for  the  rocket. 
That  is,  we  would  need  a  second  predicate  for  load  which  checks  whether  the 
rocket  has  still  free  capacity.  This  predicate  has  to  be  provided  with  the  domain 
specification:  the  test  ffull {rocket)  has  to  be  included  into  the  preconditions  of 
load. 


^Remark:  This  will  probably  done  in  the  diploma  thesis  of  Martin  Miihlpfordt. 
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Table  4.7:  The  hanoi  domain  (static  predicates  are  not  listed  for  each  state) 

:  To?er  of  Hanoi  (3  discs) 

(Hake-Package  *pd-kaitoi) 

(Export  ’(states  goal  ntles  is-c-pred  c-eq)) 

;  sseEB=sss=es=s=ss=====r===s==s===========ss=======ss=s=ss=s£sss==s 


(setq  states  ’( 
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)  )) 

;  statics  (currently  included  in  every  state) 

;(sHaller  pi  dl)  (snaller  pi  d2)  (saaller  pi  d3)  (saaller  p2  dl) 

:(saaller  p2  d2)  (saaller  p2  d3)  (saaller  p3  dl)  (saaller  p3  d2) 

; (saaller  p3  d3)  (saaller  d2  dl)  (saaller  d3  dl)  (saaller  d3  d2) 

(setq  goal  ’((on  d3  p3)  (on  d2  d3)  (on  dl  d2))) 

(setq  rules  ’( 

(rule  Bove 

(if  (on  (>  d)  (>  froa))  (ct  «  d))  (saaller  (>  to)  «  d))  (ct  «  to))) 
(tken  (aove  (<  d)  (<  froa)  «  to))) 

(conq  (add  ((on  (<  d)  (<  to))  (ct  «  froa))) 

del  ((on  (<  d)  (<  froa))  (ct  (<  to)))  )  )))) 


4,2.3  Tower  of  Hanoi 

The  domain  specification  for  hanoi  is  given  in  table  4.7.  Here  one  shortcom¬ 
ing  of  our  currently  restricted  specification  language  becomes  obvious:  static 
predicates  (i.e.  predicates  which  are  never  changed  by  an  operator  application 
“  in  contrast  to  fluents)  cannot  be  defined  globally  for  a  domain  but  they  have 
to  be  enumerated  for  each  state.  The  graphical  DPI  an  output  is  too  large  for 
one  screen-shot.  Therefore  we  will  give  a  ‘  ‘hand-drawn”  plan  tree  instead  (see 
figure  4.10).  A  recursive  program  for  solving  Tower  of  Hanoi  problems  is  given 
in  appendix  D. 


4.3  Planning  for  list  and  number  problems 

Planning  problems  -  as  blocksworld.  Tower  of  Hanoi,  transportation,  or  schedul¬ 
ing  problems  -  are  typically  not  considered  in  program  synthesis.  On  the  other 
hand,  number  and  list  problems  are  only  seldom  considered  in  planning.  Re¬ 
member,  that  our  planning  approach  is  mainly  intended  for  providing  initial 
programs  for  program  synthesis.  To  be  able  to  compare  our  approach  with 
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d2  p1  d3 
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/move  d1  d2  p2 


move  d1  d3  d2 


d2  p3  p1 


move\d1  d2  d3 


Figure  4.10:  Output  of  DPlan  for  hanoi  (abridged  drawing) 
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other  current  (ILP)  approaches  (as  FOIL  [Qui90],  GOLEM  [MF90],  or  PRO- 
GOL  [Mug95]),  we  have  to  be  able  to  deal  with  the  standard  examples  used  in 
this  area  which  are  mostly  list  processing  problems. 

For  a  lot  of  typical  planning  problems  there  exist  isomorphic  or  at  least 
structural  similar  list  and/or  number  problems  [SW98].  For  example,  append 
has  an  nearly  isomorphic  structure  to  loadob  and  unloadob-  the  only  difference 
is,  that  we  need  a  selector  (head)  and  a  reductor  (tail)  function  instead  of  only 
next: 


append{ll,l2)  =  g{empty{ll)J2jCons{hd{ll),append{tl{ll)J2))). 

The  clearblock  problem  is  similar  to  calculating  the  factorial  of  a  number 
[S War]  -  instead  of  the  parameter  s  representing  the  (globally  available)  current 
situation  we  here  have  a  constant  output  (1): 

factorial{x)  =  ^(egO(a;),  mult [x^  factor ial{pr€d{x)))). 

While  generalizing  recursive  marcos  from  initial  programs  for  these  kind  of 
problems  is  easy  [MS98],  generating  such  initial  programs  with  DPlan  is  cur- 
rently  not:  Domain  specifications  for  append  and  factorial  cannot  be  repre¬ 
sented  in  a  natural  way  in  the  current  version  of  DPlan  because  of  our  limited 
domain  specification  language^.  In  full  ADL  we  could  represent  the  factorial 
domain  as: 
mult(xjy): 

PRECOND:  x^O 
UPDATE:  a:  ^  r  x  (x  —  1) 
fixed  (x): 

PRECOND:  =  0 
UPDATE:  x^l. 

Allowing  conditional  effects,  the  operators  could  be  integrated  in  a  single  one 
with  no  global  precondition. 

We  give  an  example  for  modelling  a  list-domain  without  these  features  for 
sort  (see  table  4.8;  see  also  a  specification  of  this  domain  in  PRODIGY  in 
appendix  E).  This  domain  was  originally  introduced  in  [Wys87]. 

Again,  we  have  to  enumerate  the  static  predicates  for  each  state.  The  swap 
operator  here  is  restricted  to  list  neighbors  -  therefore,  the  intended  cyclic  macro 
is  a  bubble-sort  algorithm.  The  DPlan  output  is  given  in  figure  4.11. 


^Remark:  Extending  our  domain  specification  language  will  be  worked  on  by  Eckhard 
Wiederhold  as  his  student  project 
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Table  4.8;  The  sort  domain 


;  sort 


(Hake -Package  ’pd-sort) 

(Export  ’(states  goal  rules  is-c-pred  c-eq)) 


(setq  states  ’( 

((isc  pi  1>  (isc  p2  2)  (isc  p3  3)  (db  pi  p2)  (db  p2  p3)) 

((isc  pi  1)  (isc  p2  3)  (isc  p3  2)  (db  pi  p2)  (db  p2  p3)) 

((isc  pi  2)  (isc  p2  1)  (isc  p3  3)  (db  pi  p2)  (db  p2  p3)) 

((isc  pi  2)  (isc  p2  3)  (isc  p3  1)  (db  pi  p2)  (db  p2  p3)) 

((isc  pi  3)  (isc  p2  1)  (isc  p3  2)  (db  pi  p2)  (db  p2  p3)) 

^^((isc  pi  3)  (isc  p2  2)  (isc  p3  1)  (db  pi  p2)  (db  p2  p3)) 

;  isc  =  is-content  position  eleaent 
;  db  =  directly  before  position.i  po8itioii_j 
;  db  is  static 

(setq  goal  ’((isc  pi  1)  (isc  p2  2)  (isc  p3  3))) 

(setq  rules 
’(  (mle  snap 

(if  (db  (>  p)  (>  q))  (isc  (>  p)  (>  nl))  (isc  (>  q)  (>  n2))) 
(then  (snap  «  p)  «  q))) 

(conq  (add  ((isc  «  p)  «  n2))  (isc  «  q)  «  nl))) 
del  ((isc  «  p)  «  nl))  (isc  «  q)  «  n2))) 

)  ) 

) 

)) 


Figure  4.11;  Output  of  DPlan  for  sort 


Chapter  5 

Conclusions  and  further 
work 


We  presented  the  state-based  non-linear  backward  planner  DPlan.  DPlan  is  a 
universal  planner  for  deterministic  domains  which  is  sound  and  complete  and 
constructs  optimal  plans.  The  current  implementation  relies  on  sets  of  states 
as  input,  that  is,  planning  is  used  to  construct  a  complete  partial  order  over 
states  with  respect  to  the  number  of  operations  needed  to  transform  a  state 
into  a  state  fulfilling  the  planning  goals.  Providing  such  state  sets  results  in  an 
only  polynomial  effort  for  planning.  Because  DPlan  is  mainly  intended  as  a  tool 
for  constructing  initial  programs  as  input  into  our  program  synthesis  algorithm 
[SW98,  MS98],  we  typically  only  plan  for  small  domains  (where  it  is  easy  to 
enumerate  all  states  for  an  user).  Our  next  implementation  will  be  based  on 
the  alternative  strategy  described  in  chapter  3  -  planning  without  initial  states. 
Because  we  then  have  (1)  to  generate  all  possible  predecessors  for  each  state 
node,  and  (2)  eliminate  non-admissible  and  already  considered  states,  planning 
effort  will  than  be  (as  usual  for  generic  planners)  exponential  in  the  worst  case. 
The  next  version  of  DPlan  will  be  able  to  use  a  more  powerful  ADL-like  domain 
specification  language.  Our  main  concern  is  to  introduce  universal  quantification 
and  the  possibility  of  applying  functions. 

Secondly,  we  presented  our  approach  for  transforming  DPlan  outputs  into 
initial  programs.  The  current  implementation  can  only  deal  with  linear  plans, 
an  extension  to  non-linear  plans  is  under  construction. 

We  believe  that  combining  planning  with  program  synthesis  is  fruitful  for 
both  areas  of  research:  On  the  one  hand,  planning  provides  a  more  powerful  tool 
for  constructing  initial  (straight-forward)  programs  than  the  search  and  rewrite 
techniques  usually  employed  in  inductive  program  synthesis  systems  (see  chap¬ 
ter  2) .  Planning  is  the  natural  approach  to  model  the  search  of  transformation 
sequences  for  inputs  into  desired  outputs  and  it  provides  a  standardized  and 
powerful  language  for  representing  knowledge  about  primitive  operators.  We 
have  extended  this  language  to  represent  also  knowledge  about  data-structures 
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-  rewrite-rules  to  replace  constants  by  constructive  expressions.  On  the  other 
hand,  program  synthesis  provides  a  powerful  approach  to  cyclic  macro  general¬ 
ization.  Our  synthesis  system  [SW98]  which  is  based  on  the  theory  of  recursive 
functions  and  formalized  in  the  framework  of  grammatical  inference  [MS98]  pro¬ 
vides  a  formally  sound  and  purely  syntactical  approach  to  generalizing  recursive 
functions  from  initial  programs.  The  idea  of  cyclic  macros  can  also  be  used  to 
make  universal  planning  more  efficient:  plans  have  only  generated  for  small  fi¬ 
nite  domains  and  then  can  be  generalized  to  domains  of  arbitraty  complexity. 
Finally,  cyclic  macros  provide  an  alternative  to  learning  control  rules  [VCP+95]: 
a  cyclic  macro  which  in  our  approach  is  represented  as  recursive  program  scheme 
represents  the  complete  subgoal  structure  of  a  domain.  Therefore,  if  a  cyclic 
macro  is  available,  a  plan  can  be  generated  completely  without  search  by  simply 
the  macro  to  a  given  input  state.  To  make  cyclic  macros  available  of 
planning,  we  have  to  provide  a  method  for  selecting  an  appropriate  macro  from 
memory.  This  is  clearly  a  non-trivial  problem;  two  ideas  for  macro  retrieval  are 
discussed  at  the  beginning  of  section  4.2.  A  drawback  in  contrast  to  learning 
in  PRODIGY  [VCP'^95]  is,  that  in  our  approach,  learning  cannot  be  performed 
incremental:  To  generalize  a  cyclic  macro  information  about  the  transformation 
structure  for  the  complete  state  set  has  to  be  available. 

Clearly,  there  is  still  a  lot  to  do,  to  bring  our  approach  to  a  scope  and  a 
level  of  performance  similar  to  PRODIGY  [VCP'*~95]  —  another  system  that 
combines  planning  and  machine  learning.  But  we  strongly  believe,  that  using 
the  formal  background  of  program  synthesis  for  macro  learning  can  open  an  new 
perspective  on  learning  in  planning. 


Appendix  A 

Implementation  of  DPlan 


README  for  DPlan 
Ute  Schmid  Dec .  1998 

available  at  http://ki.cs.tu-b€rlin.de/''schmid/DPlan-l  .0/ 


DPlan  consists  of  the  following  modules : 

-  dplan.lsp  :  main  module  for  planning  and  transformation  of  dplans 

to  initial  programs 

-  plan-dstruc .Isp:  global  datastructure  (each  plemning  step  as  structure) 

-  ps-back.lsp  :  backward  application  of  sets  of  operators  to  current  state 

-  c-rewrite . Isp  :  rewriting  of  constants  to  constructive  expressions 

-  p-xtree.lsp  :  graphical  output  of  trees  via  xtree  (platform  dependent!) 

-  myxtree  :  shell-script  for  calling  xtree 

-  xtree  ;  public  domain  program,  see  HELP-install 


Currently  implemented  for  Allegro  Common  Lisp 
(previous  version  for  clisp) 

If  want  to  avoid  to  adapt  dplan  to  your  platform 
outcomment  the  following  expressions  in  dplan.lsp 

-  loading  of  package  p-xtree 

-  the  calls  of  (xtree  tree  name)  in  function  plan-control 

-  it  is  possible  that  you  have  to  outcomment  additionally 
(also  in  plan-control) 

♦  the  external  call  of  xmv  (“save  window  with  grafical  output  as  give") 

♦  or  adapt  the  syntax  for  calling  external  functions 
"user : : run- she 11 -command  ..." 

to  your  version  of  common  lisp 


Before  starting  dplan: 
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-  make  sure  that  you  give  the  correct  path  for  xtree  in  myxtree 
if  you  want  to  use  the  graphical  output 

-  provide  a  problem  specification  in  the  correct  format  (required 
as  input) 

->  see  pd-cb,  pd-tower,  pd-rocket  as  examples 


Running  the  planner: 

call  lisp,  load  dplan,  call  (start) 

(input  of  a  problem-specification  file  is  handled  interactively) 


Appendix  B 

A  recursive  program  for 
building  towers 


Building  a  tower  of  alphabetically  ordered  blocks 
in  blocksworlds  with  arbitrary  numbers  of  blocks 
and  for  arbitrary  initial  states 
Ute  Schmid  Dec  4  97 


Representation:  blocks  names  als  numbers  l...n  (instead  of  A,  B,  C...) 

each  partial  tower  as  list 
Input:  list  of  lists 
Examples  for  the  three  blocks  world 

((1  2  3))  ((2  3)  (D)  ((1)  (2)  (3))  ((2  1)  (3))  ((3  2  1))  ((1  2)  (3)) 

((1  3)  (2))  ((3  1)  (2))  ((3  2)  (D)  ((1  3  2))  ((2  3  1))  ((2  1  3))  ((3  1  2)) 


;  help  functions 


;  flattens  a  list  1 
(defun  flatten  (1) 

(cond  ((null  1)  nil) 

(T  (append  (car  1)  (flatten  (cdr  1)))) 

) 

) 


;  x+1  =  y? 

(defun  onedif  (x  y) 
(=  (1+  X)  y) 

) 


;  blocks  world  selectors 


;  topmost  block  of  a  tower 
(defun  topof  (tw) 

(car  tw) 

) 

;  bottom  block  (base)  of  a  tower 
(defun  bottom  (tw) 

(car  (last  tw)) 

) 


;  next  tower 
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;  f.e.  ((2  1)  (3))  ->  (2  1) 

(defun  get-tower  (1) 

(car  1) 

) 


;  tops  of  all  current  towers 
(defun  topelements  (1) 

(sort  (map  »list  t.’car  1)  i’>) 

) 

;  topblock  with  highest  number 
(defun  greatest  (1) 

(car  (topelements  1)) 

) 


;  topblock  mat  second  highest  number 
(defun  scndgreatest  (1) 

(cadr  (topelements  1)) 

) 

;  label  of  the  block  with  the  highest  number 
(defun  maxblock  (1) 

(cond  ((null  1)  0) 

(T  (car  (sort  (flatten  1)  #*>))) 

)) 


;  find  a  tower  (more  than  1  block)  which  is  not  correctly  sorted 
(defun  no-tower  (1) 

(cond  ((null  1)  nil) 

((and  (base  (get-tower  1)  1)  (sorted  (get-tower  1))) 

(no-tower  (cdr  1))) 

((single-block  (get-tower  1))  (no-tower  (cdr  1))) 

(T  (car  D) 

)) 

;  find  all  towers  which  are  not  correctly  sorted 
(defun  get-all-no-towers  (1) 

(cond  ((null  1)  nil) 

((and  (base  (get-tower  1)  1)  (sorted  (get-tower  1))) 

(get-all-no-towers  (cdr  1))) 

((single-block  (get-tower  1))  (get-all-no-towers  (cdr  1))) 

(T  (cons  (car  1)  (get-all-no-towers  (cdr  1)))) 


(defun  find-greatest  (max  1) 

(cond  ((null  1)  max) 

((>  (topof  max)  (topof  (car  1)))  (find-greatest  max  (cdr  1))) 
(T  (find-greatest  (car  1)  (cdr  1))) 

)) 

;  find  incorrect  tower  containing  highest  element 
(defun  greatest-no-tower  (1) 

(cond  ((null  1)  nil) 

(T  (find-greatest  (car  (get-all-no-towers  1)) 

(cdr  (get-all-no-towers  1)))) 


;  blocksworld  predicates 


;  is  tower  only  a  single  block? 
(defun  single-block  (tw) 

(=  (length  tw)  1) 

) 


;  has  tower  block  with  highest  number  as  base? 
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(defun  base  (tw  1) 

(equal  (bottom  tw)  (maxblock  1)) 

) 

;  exist  two  partial  towers  which  top  elements  differ  only  by  one? 
(defun  exist-free -neighbours  (1) 

(onedif  (scndgreatest  1)  (greatest  1)) 

) 

;  exists  a  correct  partial  tower? 

;  f.e.  (2  3)  or  (B  C) 

(defun  exists-tower  (1) 

(cond  ((null  1)  nil) 

((and  (equal  (bottom  (get-tower  1))  (maxblock  1)) 

(sorted  (get-tower  1)))  T) 

(T  (exists-tower  (cdr  1))) 

)) 


;  is  block  x  predecessor  to  top  of  a  tower? 

(defun  successor  (x  tw) 

(cond  ((null  tw)  T) 

((onedif  x  (car  tw))  T)  ;  (successor  x  (cdr  tw))) 
(T  nil) 

)) 


;  is  tower  sorted? 

(defun  sorted  (tw) 

(cond  ((null  tw)  T) 

((successor  (car  tw)  (cdr  tw))  (sorted  (cdr  tw))) 
(T  nil) 

)) 

;  exists  only  one  tower? 

(defun  single-tower  (1) 

(null  (cdr  1)) 

) 

;  goal  state? 

(defun  is-tower  (1) 

(and  (single-tower  1)  (sorted  (get-tower  1))) 

) 


;  blocksworld  operators 


;  put  X  on  y 
(defun  put  (x  y  1) 

(cond  ((null  1) 

(print  ’put)  (print  x)  (print  y) 
nil) 

((equal  (caar  1)  x)  (cond  ((not  (null  (cdar  1))) 

(append  (list  (cdar  1))  (put  x  y  (cdr  1)))) 
(T  (put  X  y  (cdr  1))))) 

•((equal  (caar  1)  y)  (cons  (cons  x  (car  1))  (put  x  y  (cdr  1)))) 

(T  (cons  (car  1)  (put  x  y  (cdr  1)))) 

)) 

;  puttable  x 
(defun  puttable  (x  1) 

(cond  ((null  1)  nil) 

((equal  (caar  1)  x)  (print  ’puttable)  (print  x) 

(cons  (list  x)  (cons  (cdar  1)  (cdr  1)))) 

(T  (cons  (car  1)  (puttable  x  (cdr  1)))) 

)) 
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;  main  function 


(defun  tower  (1) 

(cond  ((is-tower  1)  1) 

((and  (exists-tower  1) 

(exist-free-neighbours  1)) 

(tower  (put  (scndgreatest  1)  (greatest  1)  1))) 

(T  (tower  (puttable  (topof  (greatest-no-tower  1))  1))) 


)) 


Appendix  C 

Recursive  programs  for  the 
rocket  domain 


Iterative  Macro  for  the  Rocket  Domain 
(assuming  a  rocket  with  infinete  capacity) 
Ute  Schmid  11/23/98 


call  (rocket  boxes-at-A  boxes-at-B  boxes-in-Rocket  position-of-Rocket) 
cf.  (rocket  ^{bl  b2  b3)  nil  nil  ’A) 

(rocket  ’(bl)  Hb2  b3)  *A)) 

etc. 


1st  variant:  recursive  loadb  and  unload 

problem:  call  of  rocket  after  loadb  relies  on  instantiation  of 
boxes-at“A  with  nil 

alternatives:  global  variables 

loadb  returns  a  list  of  boxes-at-A  and  boxes-in-Rocket 
without  let  loadb  would  be  evaluated  twice*! 


;;  —  primitive  opns - ;; 

(defun  empty  (1) 

(null  1) 

) 

(defun  put  (e  1) 

(cons  e  1) 

) 

(defun  next-box  (1) 

(car  1) 

) 

(defun  rest-boxes  (1) 

(cdr  1) 

) 


; ;  —  drive - ; ; 
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(defun  drivG-to-B  (position) 

(cond  ((eq  position  ’A)  (print  'rocket-arrived-at-B)  »B) 
(  T  nil) 

)) 


;; - recursive  load  and  unload  —  ;; 

(defun  unload  (position  boxes-in-Rocket  boxes-at-B) 

(print  ‘(...unloading  box  ,6(next-box  boxes-in-Rocket))) 

(cond  ((empty  boxes-in-Rocket)  boxes-at-B) 

(T  (put  (next-box  boxes-in-Rocket) 

(unload  position  (rest-boxes  boxes-in-Rocket)  boxes-at-B) 

)  ) 

)) 


(defun  loadb  (boxes-at-A  boxes-in-Rocket) 

(print  ‘(...loading  box  (next -box  boxes-at-A))) 

(cond  ((empty  boxes-at-A)  boxes-in-Rocket) 

(T  (put  (next-box  boxes-at-A) 

(loadb  (rest-boxes  boxes-at-A)  boxes-in-Rocket) 

)  ) 

)) 

;;  —  ROCKET  —  ;; 

(defun  rocket  (boxes-at-A  boxes-at-B  boxes-in-Rocket  position-of-Rocket) 
(print  ‘  (boxes-at-A  ,fiboxes-at-A  boxes-at-B  ,0boxes-at-B 
boxes-in-Rocket  ,0boxes-in-RockGt 
position-of-Rocket  ,0position-of-Rocket) ) 

(cond  ((empty  boxes-at-A) 

(cond  ((empty  boxes-in-Rocket) 

(print  Mone) 

T 

) 

((eq  ’A  position-of-Rocket) 

(print  ’driving-to-dest-and-unloading) 

(unload  (drive-to-B  position-of-Rocket)  boxes-in-Rocket 
boxes-at-B) 

) 

(T  (print  ’cannot-reach-goal)  nil) 

)) 

(T 

(cond  ((eq  ’B  position-of-Rocket) 

(print  ’cannot-reach-goal) 
nil 

) 

(T 

(print  ’loading-boxes) 

(rocket  nil  boxes-at-B 

(loadb  boxes-at-A  boxes-in-Rocket) 
position-of-Rocket 

)) 

)) 

)) 

;;  Iterative  Macro  for  the  Rocket  Domain 
;;  (assuming  a  rocket  with  infinete  capacity) 

;;  Ute  Schmid  11/23/98 


call  (rocket  boxes-at-A  boxes-at-B  boxes-in-Rocket  position-of-Rocket) 
cf.  (rocket  ’ (bl  b2  b3)  nil  nil  ’A) 

(rocket  ’(bl)  >(b2  b3)  ’A)) 


etc. 
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; ;  primitive  opns  —  ; ; 

(defun  empty  (1) 

(null  1) 

) 

(defun  put  (e  1) 

(cons  e  1) 

) 

(defun  next-box  (1) 

(car  1) 

) 

(defun  rest-boxes  (1) 

(cdr  1) 

) 

; ;  —  drive  —  ; ; 

(defun  drive-to-B  (position) 

(cond  ((eq  position  ’A)  (print  »rocket-arrived-at-B)  ^B) 
(  T  nil) 

)) 


;;  —  ROCKET - ;; 

(defun  rocket  (boxes-at-A  boxes-at-B  boxes-in-Rocket  position-of-Rocket) 

(print  * (boxes-at-A  »®boxes-at-A  boxes-at-B  ,®boxes-at-B  » 

boxes-in-Rocket  ,fiboxes-in-Rocket 
position-of-Rocket  ,®position-of-Rocket) ) 

(cond  ((empty  boxes-at-A) 

(cond  ((empty  boxes-in-Rocket) 

(print  Mono) 

T 

) 

((eq  ’A  position-of-Rocket) 

(print  ’driving-to-dest) 

(rocket  boxes-at-A  boxes-at-B  boxes-in-Rocket 
(drive-to-B  position-of-Rocket)) 

) 

((eq  ’B  position-of-Rocket) 

(print  'unloading-boxes) 

(rocket  boxes-at-A  (put  (next-box  boxes-in-Rocket)  boxes-at-B) 
(rest-boxes  boxes-in-Rocket)  position-of-Rocket) 

) 

(T  (print  'cannot -reach-goal)  nil) 

)) 

(T 

(cond  ((eq  'B  position-of-Rocket) 

(print  'cannot-reach-goal) 
nil 

) 

(T 

(print  'loading-boxes) 

(rocket  (rest-boxes  boxes-at-A)  boxes-at-B 

(put  (next-box  boxes-at-A)  boxes-in-Rocket) 
position-of-Rocket) 

) 

)) 

)) 
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Appendix  D 


A  recursive  program 
Tower  of  Hanoi  with 
2irbitrary  input  states 


Towers  of  Hanoi  generalized  for  arbitrary  initial  states 
Ute  Schmid 


(1)  Load 

(2)  C  =  goal  peg 

(SETq  A  *(1  2  3)  B  NIL  C  NIL)  or 

(SETQ  A  »(1  2)  B  NIL  C  »(3))  or 

(SETQ  A  »(1)  B  NIL  C  »(1  2))  or 

(SETQ  A  >(2  3)  B  NIL  C  ’(!))  or 

etc.  for  3  discs: 

(3)  (ghanoi) 


27  poss. 


;  help  functions 

;  on:  returns  peg  on  which  a  disc  lies 

(DEFUN  ison  (disc  peg) 

(CQND  (  (NULL  peg)  NIL) 

(  (=  (CAR  peg)  disc)  T) 

(  T  (ison  disc  (cdr  peg))) 

) 

) 

(DEFUN  on  (disc  current-peg  goal-peg  inter-peg) 

(COND  (  (ison  disc  (eval  current -pe g) )  current-peg) 
(  (ison  disc  (eval  goal-peg))  goal-peg) 

(  T  inter-peg) 

) 

) 


;  interpeg:  returns  peg  which  is  neither  the  peg  on  which  the  currently 
;  regarded  disc  lies  nor  the  current  goal-peg 

(DEFUN  interpeg  (disc  peg) 

(COND  (  (or  (and  (equal  (on  disc  ’A  ’B  ’C)  *B)  (equal  peg  ’C)) 

(and  (equal  (on  disc  ’A  ’B  ’C)  ’C)  (equal  peg  ’B)) 

)  ’A) 
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) 


) 


(  (or  (and  (equal  (on  disc 

(and  (equal  (on  disc 

)  ’B) 

(  (or  (and  (equal  (on  disc 

(and  (equal  (on  disc 

)  »C) 


»A  'B  »C) 
>A  ’C) 

»A  »B  'O 
’A  ’B  »C) 


’A)  (equal  peg  ’C)) 
'C)  (equal  peg  ’A)) 

’A)  (equal  peg  ’B)) 
^B)  (equal  peg  »A)) 


;  topof:  returns  top  disc  of  a  peg 


(DEFUN  topof  (peg) 

(COKD  (  (null  (car  (eval  peg)))  nil) 
(  T  (car  (eval  peg))) 


;  clearpeg:  true,  if  no  disc  is  on  peg.  false  otherwise 

(DEFUN  clearpeg  (peg) 

(COND  (  (null  (car  (eval  peg)))  T) 

(  T  nil) 

) 

) 


;  cleartop:  true  if  no  disc  is  on  top  of  the  current  disc,  false  otherwise 
(DEFUN  cleartop  (disc) 

(COND  (  (and  (equal  (on  disc  ’A  ’B  »C)  »A)  (=  (car  A)  disc)) 

T) 

(  (and  (equal  (on  disc  »A  »B  ’C)  »B)  (=  (car  B)  disc)) 

T) 

(  (and  (equal  (on  disc  ’A  »B  »C)  »C)  (=  (car  C)  disc)) 

T) 

(  T  nil) 

) 

) 


;  movedisc:  put  disc  to  peg 

(DEFUN  movedisc  (disc  peg) 

(COND  (  (=  disc  0)  (PRINT  (LIST  »N0  ’DISC))) 

(  (equal  (on  disc  ’A  ’B  ’C)  peg) 

(PRINT  (LIST  ’Disc  disc  ’IS  ’ON  ’PEG  peg)) 

(  (OR  (clearpeg  peg)  (>  (topof  peg)  disc)) 

(PRINT 

(LIST  ’MOVE  ’DISC  disc 

’FROM  (on  disc  ’A  ’B  ’C) 

’TO  peg 

)  ) 

(SET  (on  disc  ’A  ’B  ’C)  (CDR  (eval  (on  disc  ’A  ’B  ’C)))) 
(SET  peg  (CONS  disc  (EVAL  peg))) 

(  T  (PRINT  (LIST  ’MOVING  disc  ’TO  peg  ’IMPOSSIBLE  ))) 

) 


;  move:  the  recursive  function  for  calculating  the  goal-stack  for 
;  solving  tower  of  hanoi 

(DEFUN  move  (disc  peg) 

(COND  (  (and  (=  disc  1)  (equal  (on  disc  ’A  ’B  ’C)  peg))  T  ) 

(  T  (COND 

(  (equal  (on  disc  ’A  ’B  ’C)  peg) 
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(move  (-  disc  1)  peg) 

) 

(  (and  (not  (equal  (on  disc  ’A  'B  ’C)  peg)) 

(not  (and  (cleartop  disc)  (clearpeg  peg))) 
(>  disc  D) 

(move  (-  disc  1)  (interpeg  disc  peg)) 

) 

) 

(movedisc  disc  peg) 

(COND  ((>  disc  1)  (move  (-  disc  1)  peg))) 

) 

) 

) 


(DEFUN  number-of-discs  (pi  p2  p3) 

(+  (LENGTH  pi)  (+  (LENGTH  p2)  (LENGTH  p3))) 

) 

;  move:  number  of  discs  x  goal-peg  ->  solution 
(DEFUN  ghanoi  () 

(move  (number-of-discs  ABC)  "C) 

) 
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Appendix  E 

List  Sorting  in  PRODIGY 


Prodigy  offers  the  possibility  to  use  infinite  types  and  external  functions.  I 
wanted  to  define  an  inifinite  type  number  and  an  external  function  which  returns 
the  content  of  a  list-position.  But,  when  I  tried  to  specify  the  list  sorting 
problem  with  these  features,  it  did  not  work^.  Therefore,  I  modelled  the  list¬ 
sorting  domain  in  a  standard  way:  VLsmgSome- Number  instead  of  the  built-in 
lisp-predicate  number p  and  explicitely  coding  the  is- content  predicate. 


^  A  meeting  with  Eugene  Fink,  Nov.  1998,  brought  the  result,  that  this  kind  of  specification 
is  currently  not  possible.  I  will  check  again  with  Manuela  Veloso. 
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;;;  Domain:  Lists  (flat,  over  nat-numbers) 

(create-problem-space  >lists  : current  t) 

(ptype-of  Position  : top-type) 

(ptype-of  Content  : top-type)  ;  represents  the  current  content  of  a  position 
(defun  Some-Number  ()  >(123456)) 


(operator  SWAP 

(params  <pl>  <p2>) 

(preconds 

(«pl>  POSITION) 

«p2>  POSITION) 

(<nl>  (and  CONTENT  (Some-Number))) 
(<n2>  (and  CONTENT  (Some-Number))) 

) 

(and  (directly-before  <pl>  <p2>) 
(is-content  <pl>  <nl>) 
(is-content  <p2>  <n2>)) 

) 

(effects 

() 

((del  (is-content  <pl>  <nl>)) 

(del  (is-content  <p2>  <n2>)) 

(add  (is-content  <pl>  <n2>)) 

(add  (is-content  <p2>  <nl>)) 

) 

) 

) 


Figure  E.l:  Definition  of  the  list-domain  in  PRODIGY 


example  problem 


;;;  sort  list  of  3  elements;  instantiate  vith 
; ; ;  [1  2  3]  0  swaps 
; ; ;  [132]  swap(3,2) 

; ; ;  [213]  swap (2,1) 

;;;  [231]  swap(3,l) ,swap(2,l) 

;;;  [312]  swap(3,l) ,swap(3,2) 

;;;  [321]  swap(3,2) ,swap(3,l) ,swap(2 ,1) 

(setf  (current-problem) 

(create-problem 
(name  sortl) 

(objects 

(posl  pos2  pos3  position) 

) 

(state 

(and  (directly-before  posl  pos2)  ;  static 
(directly-before  pos2  pos3)  ;  static 
(is-content  posl  3) 

(is-content  pos2  2) 

(is-content  pos3  1) 

)) 

(goal 

(and  (is-content  posl  1) 

(is-content  pos2  2) 

(is-content  pos3  3) 

)) 

)  ) 


Figure  E.2:  Specification  of  a  list-sorting  problem  in  PRODIGY 
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